Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: duckduckgo provider #347

Open
nazdridoy opened this issue Mar 22, 2025 · 14 comments
Open

Fix: duckduckgo provider #347

nazdridoy opened this issue Mar 22, 2025 · 14 comments

Comments

@nazdridoy
Copy link

nazdridoy/DDG-API/instruction.md (private)

DuckDuckGo AI Chat API Guide

API Flow

graph LR
    A[Get VQD Token & Hash] --> B[Make Chat Request]
    B -->|Continue conversation| B
    B -->|Token expired| A
Loading

Step 1: Obtaining VQD Token and Hash

Make a GET request to fetch the required authentication tokens:

GET https://duckduckgo.com/duckchat/v1/status

Headers:

accept: text/event-stream
accept-language: en-US,en;q=0.9
cache-control: no-cache
content-type: application/json
pragma: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36
origin: https://duckduckgo.com
referer: https://duckduckgo.com/
x-vqd-accept: 1

The response headers will contain:

  • x-vqd-4: Your VQD token
  • x-vqd-hash-1: Your VQD hash (might not always be present)

Step 2: Making Chat Requests

Make a POST request to the chat endpoint:

POST https://duckduckgo.com/duckchat/v1/chat

Headers:

accept: text/event-stream
accept-language: en-US,en;q=0.9
content-type: application/json
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36
origin: https://duckduckgo.com
referer: https://duckduckgo.com/
x-vqd-4: YOUR_VQD_TOKEN
x-vqd-hash-1: YOUR_VQD_HASH  (if available)

Request body:

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "Your question here"
    }
  ]
}

Available Models:

  • gpt-4o-mini (default)
  • meta-llama/Llama-3.3-70B-Instruct-Turbo
  • claude-3-haiku-20240307
  • o3-mini
  • mistralai/Mistral-Small-24B-Instruct-2501

Handling Responses

The response is a stream of Server-Sent Events (SSE), where each line begins with data: followed by a JSON object.

Example response line:

data: {"message":"Hello","created":1742651609,"id":"chatcmpl-BDtZxCme1WWKr0ryGzFha08ChtnbM","action":"success","model":"gpt-4o-mini-2024-07-18"}

The stream ends with:

data: [DONE]

To process this stream and get the complete response:

  1. Read each line of the response
  2. Parse lines starting with data: (excluding data: [DONE])
  3. Extract the message field from each JSON object
  4. Concatenate these messages to form the complete response

Maintaining Conversations

To continue a conversation, include previous messages in your next request:

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "Who is Donald Trump?"
    },
    {
      "role": "assistant",
      "content": "Donald Trump is an American businessman, television personality, and politician who served as the 45th president of the United States from January 20, 2017, to January 20, 2021..."
    },
    {
      "role": "user",
      "content": "When was he born?"
    }
  ]
}

Error Handling

The x-vqd-4 and x-vqd-hash-1 tokens expire periodically, so refresh them if you get authentication errors.

Example Implementations

cURL Example

# Step 1: Get VQD token and hash
curl -s -D headers.txt -H "accept: text/event-stream" -H "accept-language: en-US,en;q=0.9" -H "cache-control: no-cache" -H "content-type: application/json" -H "pragma: no-cache" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36" -H "origin: https://duckduckgo.com" -H "referer: https://duckduckgo.com/" -H "x-vqd-accept: 1" https://duckduckgo.com/duckchat/v1/status

# Extract VQD token and hash from headers.txt
VQD=$(grep -i "x-vqd-4:" headers.txt | cut -d' ' -f2 | tr -d '\r')
VQD_HASH=$(grep -i "x-vqd-hash-1:" headers.txt | cut -d' ' -f2 | tr -d '\r')

# Step 2: Make chat request
curl -N -H "accept: text/event-stream" -H "accept-language: en-US,en;q=0.9" -H "content-type: application/json" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36" -H "origin: https://duckduckgo.com" -H "referer: https://duckduckgo.com/" -H "x-vqd-4: $VQD" -H "x-vqd-hash-1: $VQD_HASH" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Who is Donald Trump?"}]}' https://duckduckgo.com/duckchat/v1/chat

Python Example

import requests
import json

# Step 1: Get VQD token and hash
headers = {
    "accept": "text/event-stream",
    "accept-language": "en-US,en;q=0.9",
    "cache-control": "no-cache",
    "content-type": "application/json",
    "pragma": "no-cache",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36",
    "origin": "https://duckduckgo.com",
    "referer": "https://duckduckgo.com/",
    "x-vqd-accept": "1"
}

session = requests.Session()
response = session.get("https://duckduckgo.com/duckchat/v1/status", headers=headers)
vqd_token = response.headers.get("x-vqd-4")
vqd_hash = response.headers.get("x-vqd-hash-1", "")

# Step 2: Make chat request
chat_headers = {
    "accept": "text/event-stream",
    "accept-language": "en-US,en;q=0.9",
    "content-type": "application/json",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36",
    "origin": "https://duckduckgo.com",
    "referer": "https://duckduckgo.com/",
    "x-vqd-4": vqd_token
}

if vqd_hash:
    chat_headers["x-vqd-hash-1"] = vqd_hash

data = {
    "model": "gpt-4o-mini",
    "messages": [
        {
            "role": "user",
            "content": "Who is Donald Trump?"
        }
    ]
}

response = session.post(
    "https://duckduckgo.com/duckchat/v1/chat",
    headers=chat_headers,
    json=data,
    stream=True
)

# Process streaming response
full_message = ""
for line in response.iter_lines():
    if line:
        line = line.decode('utf-8')
        if line.startswith('data:') and line != 'data: [DONE]':
            try:
                data = json.loads(line[5:].strip())
                if "message" in data and data["message"]:
                    full_message += data["message"]
                    print(data["message"], end="", flush=True)
            except json.JSONDecodeError:
                continue

print("\n\nFull message:", full_message)

JavaScript Example

async function chatWithDuckDuckGo(question) {
    // Step 1: Get VQD token and hash
    const statusResponse = await fetch("https://duckduckgo.com/duckchat/v1/status", {
        headers: {
            "accept": "text/event-stream",
            "accept-language": "en-US,en;q=0.9",
            "cache-control": "no-cache",
            "content-type": "application/json",
            "pragma": "no-cache",
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36",
            "origin": "https://duckduckgo.com",
            "referer": "https://duckduckgo.com/",
            "x-vqd-accept": "1"
        }
    });
    
    const vqdToken = statusResponse.headers.get("x-vqd-4");
    const vqdHash = statusResponse.headers.get("x-vqd-hash-1") || "";
    
    // Step 2: Make chat request
    const chatHeaders = {
        "accept": "text/event-stream",
        "accept-language": "en-US,en;q=0.9",
        "content-type": "application/json",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36",
        "origin": "https://duckduckgo.com",
        "referer": "https://duckduckgo.com/",
        "x-vqd-4": vqdToken
    };
    
    if (vqdHash) {
        chatHeaders["x-vqd-hash-1"] = vqdHash;
    }
    
    const response = await fetch("https://duckduckgo.com/duckchat/v1/chat", {
        method: "POST",
        headers: chatHeaders,
        body: JSON.stringify({
            model: "gpt-4o-mini",
            messages: [
                {
                    role: "user",
                    content: question
                }
            ]
        })
    });
    
    // Process the stream
    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    let fullMessage = "";
    
    while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        
        const chunk = decoder.decode(value);
        const lines = chunk.split("\n");
        
        for (const line of lines) {
            if (line.startsWith("data:") && line !== "data: [DONE]") {
                try {
                    const data = JSON.parse(line.substring(5).trim());
                    if (data.message) {
                        fullMessage += data.message;
                        console.log(data.message);
                    }
                } catch (e) {
                    // Skip invalid JSON
                }
            }
        }
    }
    
    return fullMessage;
}

// Example usage
chatWithDuckDuckGo("Who is Donald Trump?").then(message => {
    console.log("Complete message:", message);
});
@aandrew-me
Copy link
Owner

Well if you check the x-vqd-hash-1 header, it contains base64 encoded js code :)

@nazdridoy
Copy link
Author

oh yeah.. didn't notice that... maybe we can creates a client-side hash when the server doesn't provide one

@aandrew-me
Copy link
Owner

aandrew-me commented Mar 22, 2025

Well for now its working without any hash, i don't know how they compute hashes on their site, you will have to check their js code for that, I guess

@nazdridoy
Copy link
Author

rather than using random values for x-vqd-4: and x-vqd-hash-1:, why not retrieve them from GET https://duckgo.com/duckchat/v1/status? this is how it works in their web UI.

@aandrew-me
Copy link
Owner

The app isn't using random values

@nazdridoy
Copy link
Author

only for hash maybe

headers["x-vqd-hash-1"] = "abcdefg"

Anyway, so far I've seen that using x-vqd-hash-1 from the status response works. If tgpt works without any hash, that's great. I haven't been able to try it yet, since it hasn't been updated in the Arch repo.

@nazdridoy
Copy link
Author

tgpt DDG is broken again; maybe this can help?

@aandrew-me
Copy link
Owner

Hmm it doesn't work

@nazdridoy
Copy link
Author

i've tried fixing tgpt, and it's working now, but chat continuation still doesn't work.

in the gpt4free (Python) project, this fix works.

❯ tgpt --provider openai --url "http://localhost:1337/api/DDG/chat/completions" --model "o3-mini" hi
          
Hello! How can I assist you today?

@nazdridoy
Copy link
Author

nazdridoy commented Mar 27, 2025

In the meantime, I'm trying to figure out how they generate the client hash. I think I've mostly figured out how it works.

sequenceDiagram
    participant C as Client
    participant S as Server
    
    Note over C,S: Authentication Flow
    
    C->>+S: GET /duckchat/v1/status
    S-->>-C: Response Headers:<br/>• x-vqd-4<br/>• x-vqd-hash-1 (base64 encoded)
    
    Note over C: Decode x-vqd-hash-1<br/>(base64 decode)
    
    Note over C: Generate client hash<br/>using response hash
    
    Note over C: Create POST payload:<br/>• Array 1: Server hash<br/>• Array 2: Client hash<br/>(both base64 decoded)
    
    C->>+S: POST /duckchat/v1/chat<br/>Headers: x-vqd-hash-1
    S-->>-C: Response


Loading

DDG client hash generation(messy notes)

DDG client hash generation(messy notes)
(function () {
  return {
        server_hashes: ["vYj6hsWWTchDO3IXFrpngPddPRcHwCYHO/mpN+mkImc=","TItQYWEYyMCYaunWovmHN33r67rb9SUTbGQeAqeujKE="],
        client_hashes: [navigator.userAgent + (navigator.userAgentData ? navigator.userAgentData.brands.map(brand => `"${brand.brand}";v="${brand.version}"`).join(', ') : ''),(function(){const e = document.createElement('div');e.innerHTML = '<div><div></div><div></div';return String(101 + e.innerHTML.length);})()],
        signals: {}
    };
})()




const result = (function () {
    return {
        server_hashes: ["MZeO1rN3+EphrLyzM8hEoAIncpk+40pv1EyLrlGWJJE=","ukJU198teAczYKtLiGI9R62lttWRHIdzKvIKhIzfbFc="],
        client_hashes: [
            navigator.userAgent + (navigator.userAgentData ? navigator.userAgentData.brands.map(brand => `"${brand.brand}";v="${brand.version}"`).join(', ') : ''),
            (function() {
                const e = document.createElement('div');
                e.innerHTML = '<div><div></div><div></div';
                return String(3393 + e.innerHTML.length);
            })()
        ],
        signals: {}
    };
})();

console.log(result.client_hashes);





{
"server_hashes":["kXtEAXHnC8ngOLACsX8RHUBtdz93TZ/+cz1e7aeehGc=","i1IgsmXoCsdo2Mur54pxFX5p8rwaI4S+TboMYshkU94="],
"client_hashes":["1IiuLsAouAVDiqKmlX+Y86x5fbiGXweBTXOUBwKWrvs=","v6u2zwLapr8dnpHF0UH1Xji4Rgr0A7wKQ3JqjqtHWFU="],"signals":{}}


{
    "input": [
        "Mozilla/5.0 (X11; Linux x86_64; rv:136.0) Gecko/20100101 Firefox/136.0",
        "3426"
    ],
    "client_hashes": [
        "1IiuLsAouAVDiqKmlX+Y86x5fbiGXweBTXOUBwKWrvs=",
        "TF+GOieSINg5xUfwMw+OijvQ+SdSoL1/n2A6azoza/U="
    ]
}


(async () => {
  const hashString = async (input) => {
    const encoder = new TextEncoder();
    const data = encoder.encode(input);
    const hashBuffer = await crypto.subtle.digest('SHA-256', data);
    const hashArray = new Uint8Array(hashBuffer);
    const hashString = Array.from(hashArray)
      .map(byte => String.fromCharCode(byte))
      .join('');
    return btoa(hashString);
  };

  const clientHashes = [
    "Mozilla/5.0 (X11; Linux x86_64; rv:136.0) Gecko/20100101 Firefox/136.0",
    "3426"
  ];

  const hashedValues = await Promise.all(clientHashes.map(str => hashString(str)));
  
  console.log({
    "input": clientHashes,
    "client_hashes": hashedValues
  });
})();

const e = document.createElement('div');
e.innerHTML = '<div><div></div><div></div';
const hashValue = String(101 + e.innerHTML.length);
console.log(hashValue);






(function () {
  return {
        server_hashes: ["RvarLp3KDHoq62Hxyw8VFxZsqJVPU5iHB/Fg0yHVZpk=","hcpqmi1rdGyBN9cavGF/s+zdbOulBanh6L2BxZA/YOE="],
        client_hashes: [navigator.userAgent + (navigator.userAgentData ? navigator.userAgentData.brands.map(brand => `"${brand.brand}";v="${brand.version}"`).join(', ') : ''),(function(){const e = document.createElement('div');e.innerHTML = '<p><div></p><p></div';return String(7728 + e.innerHTML.length);})()],
        signals: {}
    };
})()

const e = document.createElement('div');
e.innerHTML = '<p><div></p><p></div';
const hashValue = String(7728 + e.innerHTML.length);
console.log(hashValue);

7760

[ "1IiuLsAouAVDiqKmlX+Y86x5fbiGXweBTXOUBwKWrvs=", "cr302oumqbbldEmkjsj1xYRNskrpw+UbGfGPMEezOSU=" ]

{"server_hashes":["RvarLp3KDHoq62Hxyw8VFxZsqJVPU5iHB/Fg0yHVZpk=","hcpqmi1rdGyBN9cavGF/s+zdbOulBanh6L2BxZA/YOE="],"client_hashes":["1IiuLsAouAVDiqKmlX+Y86x5fbiGXweBTXOUBwKWrvs=","cr302oumqbbldEmkjsj1xYRNskrpw+UbGfGPMEezOSU="],"signals":{}}
eyJzZXJ2ZXJfaGFzaGVzIjpbIlJ2YXJMcDNLREhvcTYySHh5dzhWRnhac3FKVlBVNWlIQi9GZzB5SFZacGs9IiwiaGNwcW1pMXJkR3lCTjljYXZHRi9zK3pkYk91bEJhbmg2TDJCeFpBL1lPRT0iXSwiY2xpZW50X2hhc2hlcyI6WyIxSWl1THNBb3VBVkRpcUttbFgrWTg2eDVmYmlHWHdlQlRYT1VCd0tXcnZzPSIsImNyMzAyb3VtcWJibGRFbWtqc2oxeFlSTnNrcnB3K1ViR2ZHUE1FZXpPU1U9Il0sInNpZ25hbHMiOnt9fQ==


4-323184417266863052404559269421880207118


HTTP/2 200 
server: nginx
date: Wed, 26 Mar 2025 23:32:47 GMT
content-type: application/json; charset=utf-8
content-length: 14
etag: W/"e-BBQeGsEgpElDZSMrc0GIzke09hU"
strict-transport-security: max-age=31536000
permissions-policy: interest-cohort=()
x-frame-options: SAMEORIGIN
x-xss-protection: 1;mode=block
x-content-type-options: nosniff
referrer-policy: origin
expect-ct: max-age=0
expires: Wed, 26 Mar 2025 23:32:46 GMT
cache-control: no-cache, no-store
X-Firefox-Spdy: h2

@aandrew-me
Copy link
Owner

I was able to implement their hash generation, however i am still getting errors.
ca0d215

@nazdridoy
Copy link
Author

nazdridoy commented Mar 27, 2025

ca0d215#diff-8860bac4854ca37cd04b6daa0a75d8b33e9397fd67e2a97e4299f35e3e7efd86R84-R86

"client_hashes": []interface{}{`Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"Chromium";v="134", "Not:A-Brand";v="24", "Brave";v="134"`, "6823"},

second element of the client_hashes is dynamically calculated as e.innerHTML changes every time

e.innerHTML = '<p><div></p><p></div';return String(7728 + e.innerHTML.length)
e.innerHTML = '<div><div></div><div></div';return String(101 + e.innerHTML.length)

@Cris70
Copy link

Cris70 commented Apr 8, 2025

Just updated to 2.9.4 but I'm still unable to use duck.ai as provider

@aandrew-me
Copy link
Owner

Just updated to 2.9.4 but I'm still unable to use duck.ai as provider

Well it hasn't been fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants