How to Use SSE in AI Chat (Step-by-Step Guide)

Explore SSE in AI chat apps: advantages over WebSocket, implementation with EventSource and Fetch APIs. Learn efficient real-time communication for AI chats.

Apr 13, 2024 · 2 min read · 300 Words · -Views -Comments · Programming
WebShell SSE Fetch 🇨🇳 阅读中文版
Read with AI

Choose an AI assistant

ChatGPT Open in a new tab with the article link and prompt
Claude Copy the prompt, then open Claude

Open Claude
Trouble reading the article?

If the prompt is missing, copy the full prompt below into your AI assistant. If the AI cannot read the article, paste the visible passage you want to discuss and add your question.

ChatGPT has been popular for a year now。It has also driven the use of SSE.

Here, I summarize my understanding of it.

https://static.1991421.cn/2024/2024-04-13-153102.jpeg

Advantages of SSE

  1. Uses the standard HTTP protocol, and not WebSocket. Relative to WebSocket, it has a smaller resource overhead.
  2. SSE transmits text data with simple overhead.

How to Use

  1. EventSource

    const evtSource = new EventSource("sse.php");
    evtSource.onmessage = (e) => {
      console.log(`message: ${e.data}`);
    };
    
  2. Fetch

    fetch('/api/openai/stream', {
          method: 'post',
          headers: {
            'Content-Type': 'application/json;charset=utf-8',
          },
          body: JSON.stringify({
            messages
          })
        }).then(res => {
          const reader = res.body.getReader();
          let buffer = '';
          const readChunk = () => {
            reader.read()
              .then(({
                value,
                done
              }) => {
                if (done) {
                  console.log('Stream finished');
                  return;
                }
    
                const chunkString = new TextDecoder().decode(value);
                buffer += chunkString;
    
                let position;
                while ((position = buffer.indexOf('\n\n')) !== -1) {
                  const completeMessage = buffer.slice(0, position);
                  buffer = buffer.slice(position + 2);
                  completeMessage.split('\n').forEach(line => {
                    if (line.startsWith('data:')) {
                      const jsonText = line.slice(5).trim(); 
                      if (jsonText === '[DONE]') {
                        console.log('done');
                        return;
                      }
                      try {
                        const dataObject = JSON.parse(jsonText);
                        console.log(dataObject);
                      } catch (error) {
                        console.error('JSON parse error:', error, jsonText);
                      }
                    }
                  });
                }
                readChunk();
              })
              .catch(error => {
                console.error(error);
              });
          };
          readChunk();
        })
    

Fetch vs EventSource

  1. Both have roughly the same compatibility.
  2. A disadvantage of EventSource is that it cannot send a request body, so it typically requires setting up a request to send messages first, then initiating EventSource. Fetch, like XHR, can carry a request body. Thus, I think Fetch is the better choice.

ChatGPT’s Effect

ChatGPT has long supported stream returns, which give users the effect of a typewriter. Here is how GPT does it:

ChatGPT uses Fetch, send request body, and the return type is text/event-stream; charset=utf-8, meaning the server continuously returns response chunk until it ends.

You can find requests can be found in the network, and the request url is https://chat.openai.com/backend-api/conversation

done.

Authors
Developer, digital product enthusiast, tinkerer, sharer, open source lover