Transformer RNN modelsLucidworks AI custom embedding model training

Lucidworks AI supports general types of transformer with recurrent neural network (RNN) head models.

This architecture in Lucidworks AI is a RNN model that uses a frozen pre-trained transformer (FPT) base. The RNN head is trained while utilizing the frozen transformer weight, and captures the generality that RNN without frozen weights might not achieve.

This hybrid architecture is larger, which increases training time compared to RNN-only architecture, but produces higher-quality embeddings. Use Transformer RNN models for general use cases.

Lucidworks recommends you use RNN models for ecommerce use cases due to slower model encoding speed and less flexibility for domain specific language.