Context
A client held business data and documents from several sources that were hard to use and query. The engagement, carried out as part of a team, aimed to make them usable by production AI processing. The client is not named (confidentiality).
Problem
Collect and structure heterogeneous data, then enable reliable semantic search and generative-AI processing over business documents, with services exposed through APIs.
Approach
- Python ETL pipelines to collect, transform and integrate data from several sources.
- RAG pipeline: vectorisation of business documents, Hugging Face Transformers embeddings and a FAISS vector index for contextual retrieval.
- Integration and adaptation of the Mistral 7B model for AI processing on domain-specific business data.
- Python modules for automatic text processing, document structuring and content indexing.
- Backend services and REST APIs with FastAPI; transformed data loaded into PostgreSQL with a suitable relational structure.
- Contribution to evaluating, tuning and optimising the performance of the AI pipelines.
Stack
- Python
- ETL/ELT
- RAG
- Hugging Face
- FAISS
- Mistral 7B
- PyTorch
- FastAPI
- PostgreSQL
- React
- Git
Status
Client engagement
Engagement delivered through ORIA, as part of a team; client details not published.
What I learned
A useful RAG depends as much on document preparation — chunking, structuring, indexing — as on the choice of model. And an open model such as Mistral 7B keeps data control on the client side.