Skip to content

community: add ChatLLM7 integration #30765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

community: add ChatLLM7 integration #30765

Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6340bdb
feat: add ChatLLM7
chigwell Apr 10, 2025
2760253
fix: update `make` formatting for llm7.ipynb
chigwell Apr 10, 2025
f875667
fix: update "API reference" header
chigwell Apr 10, 2025
efd52b4
feat: add npm package link
chigwell Apr 10, 2025
8496c27
Merge branch 'master' into add-chat-llm7
chigwell Apr 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
307 changes: 307 additions & 0 deletions docs/docs/integrations/chat/llm7.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,307 @@
{
"cells": [
{
"cell_type": "raw",
"id": "87a2c4f1",
"metadata": {
"id": "87a2c4f1"
},
"source": [
"---\n",
"sidebar_label: LLM7\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "a1b3d5e6",
"metadata": {
"id": "a1b3d5e6"
},
"source": [
"# ChatLLM7\n",
"\n",
"This documentation will help you get started with LLM7's [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatLLM7` features and configurations, see the [API reference](#api-reference). The `ChatLLM7` class connects to LLM7's API-compatible endpoints.\n",
"\n",
"## Overview\n",
"### Integration details\n",
"\n",
"| Class | Package | Local | Serializable | [JS support](https://www.npmjs.com/package/langchain-llm7) | Package downloads | Package latest |\n",
"| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
"| [ChatLLM7](https://github.com/chigwell/langchain_llm7/blob/main/langchain_llm7/chat_llm7.py#L37) | [langchain_llm7](https://pypi.org/project/langchain-llm7/) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-llm7?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-llm7?style=flat-square&label=%20) |\n",
"\n",
"### Model features\n",
"| Tool calling | Structured output | JSON mode | Image input | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | Logprobs |\n",
"| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",
"| ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n",
"\n",
"## Setup\n",
"\n",
"### Installation\n",
"\n",
"Install the LangChain LLM7 integration package:\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d4e5f6a7",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "d4e5f6a7",
"outputId": "b7c0059c-a408-4147-b652-b6786e323573"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m65.2/65.2 kB\u001b[0m \u001b[31m1.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m423.3/423.3 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m443.6/443.6 kB\u001b[0m \u001b[31m21.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h"
]
}
],
"source": [
"%pip install -qU langchain-llm7"
]
},
{
"cell_type": "markdown",
"id": "b2c3d4e5",
"metadata": {
"id": "b2c3d4e5"
},
"source": [
"## Instantiation\n",
"\n",
"Create a ChatLLM7 instance with your preferred parameters:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "e6c38580",
"metadata": {
"id": "e6c38580"
},
"outputs": [],
"source": [
"from langchain_llm7 import ChatLLM7\n",
"\n",
"llm = ChatLLM7(\n",
" model_name=\"gpt-4o-mini-2024-07-18\",\n",
" temperature=0.7,\n",
" max_tokens=300,\n",
" timeout=90,\n",
" streaming=True,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d5e6f7a8",
"metadata": {
"id": "d5e6f7a8"
},
"source": [
"## Invocation\n",
"\n",
"Basic chat completion example:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "a1b2c3d4",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "a1b2c3d4",
"outputId": "3a99a7a8-2906-44c1-d21e-5a26eca3565f"
},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Quantum physics, also known as quantum mechanics, is the branch of physics that studies the behavior of matter and energy at very small scales, such as atoms and subatomic particles. Here are some key concepts explained in simple terms:\\n\\n1. **Quantization**: Energy and other properties are not continuous but come in small, discrete amounts called \"quanta.\" For example, light can be thought of as being made up of tiny packets of energy called photons.\\n\\n2. **Wave-Particle Duality**: Particles, such as electrons, can exhibit properties of both particles and waves. This means they can behave like a small ball (particle) in some situations and like ripples on water (wave) in others.\\n\\n3. **Uncertainty Principle**: Proposed by Werner Heisenberg, this principle states that you cannot precisely know both the position and momentum of a particle at the same time. The more accurately you know one, the less accurately you can know the other.\\n\\n4. **Superposition**: Particles can exist in multiple states or configurations at once until they are observed. For example, an electron in an atom can be in several energy levels at the same time until a measurement is made.\\n\\n5. **Entanglement**: When particles become entangled, the state of one particle is linked to the state of another, no matter how far apart they are. Changing the state of one instantaneously affects the other, even across vast distances.\\n\\nIn summary, quantum physics reveals a strange and counterintuitive world where particles behave unpredictably, and our observations can influence what we see. It challenges our classical notions of how things should behave and has profound implications for technology, including quantum computing and cryptography.', additional_kwargs={}, response_metadata={}, id='run-3c71820b-8bf0-401f-ad2b-2efdb3a33a7a-0', usage_metadata={'input_tokens': 18, 'output_tokens': 305, 'total_tokens': 323})"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_core.messages import HumanMessage, SystemMessage\n",
"\n",
"messages = [\n",
" SystemMessage(\n",
" content=\"You are a physics tutor. Explain concepts clearly and concisely.\"\n",
" ),\n",
" HumanMessage(content=\"Explain quantum physics in simple terms\"),\n",
"]\n",
"\n",
"response = llm.invoke(messages)\n",
"response"
]
},
{
"cell_type": "markdown",
"id": "e7f8g9h0",
"metadata": {
"id": "e7f8g9h0"
},
"source": [
"## Chaining\n",
"\n",
"Create a translation chain with a prompt template:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "i1j2k3l4",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "i1j2k3l4",
"outputId": "00a561f6-1033-475a-c041-90f5de1474eb"
},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Sure! Let\\'s break down quantum entanglement in a way that\\'s easy to understand.\\n\\n### What is Quantum Entanglement?\\n\\nQuantum entanglement is a fascinating phenomenon that occurs when two or more particles become connected in such a way that the state of one particle is directly related to the state of the other, no matter how far apart they are. It’s like the particles are communicating with each other instantaneously, even if they are light-years apart!\\n\\n### How Does It Happen?\\n\\n1. **Creation of Entangled Particles**: \\n Imagine you have a pair of dice that are special: whenever you roll one die, the other die will always show the same number, no matter where they are. This is similar to how particles become entangled. They often become entangled during interactions, like when two particles collide and then separate.\\n\\n2. **Measurement**:\\n Once the particles are entangled, if you measure one particle (for example, checking its spin or polarization), you instantly know the state of the other particle. It’s as if you rolled one die and immediately knew what number the other die showed, even if it’s far away.\\n\\n3. **Instant Connection**:\\n This \"connection\" between the particles is what makes entanglement so strange. If one particle is affected by something (like a measurement), it instantly affects its entangled partner, regardless of the distance separating them. This baffled even Einstein, who referred to it as \"spooky action at a distance.\"\\n\\n### Why Is It Important?\\n\\n1. **Quantum Computing**: Entanglement is crucial for quantum computers, which use principles of quantum mechanics to process information in ways that traditional computers cannot. Entangled particles can be used to perform complex calculations much more efficiently.\\n\\n2. **Quantum Cryptography**: It also plays a role in secure communication methods. Because unmeasured entangled particles remain connected, any attempt to intercept the communication would disturb the entanglement and reveal that someone is snooping.\\n\\n3. **Understanding the Universe**: Studying entanglement helps scientists understand the fundamental laws of physics and the nature of reality itself, raising questions about what happens at the quantum level.\\n\\n### Conclusion\\n\\nIn summary, quantum entanglement is a remarkable connection between particles that transcends distance. Once entangled, the particles act as a single system, and measuring one instantly gives you information about the other. It challenges our traditional notions of space and time and opens up exciting realms in technology and science!', additional_kwargs={}, response_metadata={}, id='run-3b5dd724-f1e3-4d30-b837-6a828d62ebd3-0', usage_metadata={'input_tokens': 24, 'output_tokens': 438, 'total_tokens': 462})"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_core.prompts import ChatPromptTemplate\n",
"\n",
"prompt = ChatPromptTemplate.from_messages(\n",
" [\n",
" (\n",
" \"system\",\n",
" \"You are a scientific explainer. Break down {concept} for a {audience} audience.\",\n",
" ),\n",
" (\"human\", \"Explain: {query}\"),\n",
" ]\n",
")\n",
"\n",
"chain = prompt | llm\n",
"chain.invoke(\n",
" {\n",
" \"concept\": \"quantum entanglement\",\n",
" \"audience\": \"high school\",\n",
" \"query\": \"What happens when particles become entangled?\",\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "m5n6o7p8",
"metadata": {
"id": "m5n6o7p8"
},
"source": [
"## Streaming\n",
"\n",
"Real-time token streaming example:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "q9r0s1t2",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "q9r0s1t2",
"outputId": "65107832-3865-4a44-ad01-99715e527f28"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Quantum physics, also known as quantum mechanics, is the branch of physics that deals with the behavior of very small particles, like atoms and subatomic particles (electrons, protons, etc.). Here are some key concepts explained simply:\n",
"\n",
"1. **Wave-Particle Duality**: Particles can behave both like particles and waves. For example, electrons can show interference patterns (a wave property) but also can be detected as distinct particles.\n",
"\n",
"2. **Quantization**: Energy levels in atoms are quantized, meaning an electron can only exist in specific energy levels, not in between. When an electron jumps between these levels, it absorbs or emits a specific amount of energy (usually in the form of light).\n",
"\n",
"3. **Uncertainty Principle**: Proposed by Werner Heisenberg, this principle states that you cannot precisely know both the position and momentum of a particle at the same time. The more accurately you know one, the less accurately you can know the other.\n",
"\n",
"4. **Superposition**: Particles can exist in multiple states at once until they are observed or measured. For example, a particle can be in multiple places at the same time.\n",
"\n",
"5. **Entanglement**: When two particles become entangled, the state of one particle is directly related to the state of another, no matter the distance between them. Changing the state of one instantly affects the other.\n",
"\n",
"6. **Probability**: Instead of giving exact predictions, quantum physics deals with probabilities. It can tell you how likely it is for a particle to be found in a certain state or position.\n",
"\n",
"In summary, quantum physics challenges our classical intuition and introduces a world where particles can behave in strange and non-intuitive ways, governed by probabilities rather than certainties."
]
}
],
"source": [
"for chunk in llm.stream(messages):\n",
" print(chunk.content, end=\"\", flush=True)"
]
},
{
"cell_type": "markdown",
"id": "u3v4w5x6",
"metadata": {
"id": "u3v4w5x6"
},
"source": [
"## API reference\n",
"\n",
"For full documentation of all ChatLLM7 parameters and methods, see the [API reference](https://llm7.io/).\n",
"\n",
"Key configuration options:\n",
"- `base_url`: API endpoint (default: https://api.llm7.io/v1)\n",
"- `model_name`: Model version selector (see the [models list](https://api.llm7.io/v1/models))\n",
"- `temperature`: Creativity control (0.0-2.0)\n",
"- `max_tokens`: Response length limit\n",
"- `streaming`: Enable/disable real-time streaming"
]
}
],
"metadata": {
"colab": {
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading