README.md

SWIRL AI Connect

Bring AI to the Data, not the Data to the AI.

SWIRL AI Connect is AI Infrastructure Software. We aim to build a safe, secure and robust AI Infrastructue for the enterprise.

Start Searching · Slack · Key Features · Contribute · Documentation · Connectors

Get your AI up and running in minutes, not months. SWIRL AI Connect is an open-source platform that simplifies AI infrastructure setup and empowers Retrieval Augmented Generation (RAG) without the complexity of additional tools.

Why SWIRL AI Connect?

• Instant AI: Deploy your AI solutions rapidly with built-in security controls (data compliance, firewall, granular access, etc.).
• RAG Made Easy: Perform Retrieval Augmented Generation directly on your data, eliminating the need for vector databases, LangChain, or LlamaIndex.
• No Data Movement: No ETL, re-indexing, or data movement. Work with your data where it lives.
• Boost Productivity: Enhance your team's efficiency by finding information faster and streamlining workflows.

SWIRL's Ranking in Action

SWIRL AI Connect Features

Based on SWIRL AI Connect

🔎 How Swirl Works

SWIRL AI Connect adapts and distributes user queries to anything with a search API - search engines, databases, noSQL engines, cloud/SaaS services, data siloes, etc. and uses Reader LLMs to re-rank the unified results without extracting or indexing anything.

1. Connect: Point SWIRL AI Connect to your data sources (databases, documents, etc.) adding in your auth/bearer token.
2. Query: Ask questions or provide prompts in natural language.
3. Get Results: SWIRL AI Connect combines powerful search with generative AI to deliver precise answers augmented by relevant context.

🔌 List of Connectors

Full list of connectors is available here.

For Enterprise Support on Connectors Contact the Swirl Team at: support@swirl.today

🔥 Try Swirl Now In Docker

Prerequisites

• To run Swirl in Docker, you must have the latest Docker app for MacOS, Linux, or Windows installed and running locally.

• Windows users must also install and configure either the WSL 2 or the Hyper-V backend, as outlined in the System Requirements for installing Docker Desktop on Windows.

Start Swirl in Docker

Warning Make sure the Docker app is running before proceeding!

• Download the YML file: https://raw.githubusercontent.com/swirlai/swirl-search/main/docker-compose.yaml

curl https://raw.githubusercontent.com/swirlai/swirl-search/main/docker-compose.yaml -o docker-compose.yaml

• Optional: To enable Swirl's Real-Time Retrieval Augmented Generation (RAG) in Docker, run the following commands from the Console using a valid OpenAI API key:

export MSAL_CB_PORT=8000
export MSAL_HOST=localhost
export OPENAI_API_KEY=‘<your-OpenAI-API-key>’

🔑 Check out OpenAI's YouTube video if you don't have an OpenAI API Key.

• In MacOS or Linux, run the following command from the Console:

docker-compose pull && docker-compose up

• In Windows, run the following command from PowerShell:

docker compose up

After a few minutes the following or similar should appear:

• Open this URL with a browser: http://localhost:8000 (or http://localhost:8000/galaxy)

• If the search page appears, click Log Out at the top, right. The Swirl login page will appear.

• Enter the username admin and password password, then click Login.

• Enter a search in the search box and press the Search button. Ranked results appear in just a few seconds:

• To view the raw JSON, open http://localhost:8000/swirl/search/

The most recent Search object will be displayed at the top. Click on the result_url link to view the full JSON Response.

Notes 📝

Warning The Docker version of Swirl does not retain any data or configuration when shut down!

🔑 Swirl includes five (5) Google Programmable Search Engines (PSEs) to get you up and running right away. The credentials for these are shared with the Swirl Community.

🔑 Using Swirl with Microsoft 365 requires installation and approval by an authorized company Administrator. For more information, please review the M365 Guide or contact us.

Next Steps 👇

• Check out the details of our latest release!

• Head over to the Quick Start Guide and install Swirl locally!

🌟 Key Features

✦	Feature
📌	Microsoft 365 integration and OAUTH2 support
🔍	SearchProvider configurations for all included Connectors. They can be organized with the active, default and tags properties.
✏️	Adaptation of the query for each provider such as rewriting NOT term to -term, removing NOTted terms from providers that don't support NOT, and passing down the AND, + and OR operators.
⏳	Synchronous or asynchronous search federation via APIs
🛎️	Optional subscribe feature to continuously monitor any search for new results
🛠️	Pipelining of Processor stages for real-time adaptation and transformation of queries, responses and results
🗄️	Results stored in SQLite3 or PostgreSQL for post-processing, consumption and/or analytics
➡️	Built-in Query Transformation support, including re-writing and replacement
📖	Matching on word stems and handling of stopwords via NLTK
🚫	Duplicate detection on field or by configurable Cosine Similarity threshold
🔄	Re-ranking of unified results using Cosine Vector Similarity based on spaCy's large language model and NLTK
🎚️	Result mixers order results by relevancy, date or round-robin (stack) format, with optional filtering of just new items in subscribe mode
📄	Page through all results requested, re-run, re-score and update searches using URLs provided with each result set
📁	Sample data sets for use with SQLite3 and PostgreSQL
✒️	Optional spell correction using TextBlob
⌛	Optional search/result expiration service to limit storage use
🔌	Easily extensible Connector and Mixer objects

👩‍💻 Contributing to Swirl

Do you have a brilliant idea or improvement for Swirl? We're all ears, and thrilled you're here to help!

🔗 Get Started in 3 Easy Steps:

1. Connect with Fellow Enthusiasts - Jump into the Swirl Slack Community and share your ideas. You'll find a welcoming group of Swirl enthusiasts and team members eager to assist and collaborate.
2. Branch It Out - Always branch off from the develop branch with a descriptive name that encapsulates your idea or fix.
3. Start Your Contribution - Ready to get your hands dirty? Make sure all contributions come through a GitHub pull request. We roughly follow the Gitflow branching model, so all changes destined for the next release should be made to the develop branch.

📚 First time contributing on GitHub? No worries, the GitHub documentation has you covered with a great guide on contributing to projects.

💡 Every contribution, big or small, makes a difference. Join us in shaping the future of Swirl!

☁ Use the Swirl Cloud

For information about Swirl as a managed service, please contact us!

📖 Documentation

Overview | Quick Start | User Guide | Admin Guide | M365 Guide | Developer Guide | Developer Reference | AI Guide

👷‍♂️ Need Help? We're Here for You!

At Swirl, every user matters to us. Whether you're a beginner finding your way or an expert with feedback, we're here to support, listen, and help. Don't hesitate to reach out to us.

• 🎉 Join the Conversation: Dive into our vibrant Swirl Community on Slack - it's where all the magic happens!

• 📧 Direct Support: For any questions, suggestions, or even a simple hello, drop us an email at support@swirl.today. We cherish every message and promise to get back to you promptly!

• 💼 Request A Connector (Enterprise Support) Want to see a new connector quickly and fast. Contact the Swirl Team at: support@swirl.today

Remember, you're part of our family now. 🌍💙