Spring Boot integrates sse to implement chatgpt streaming interaction
1. What is SSE?
SSE (Server-Sent Events) is a technology that allows a server to push real-time data to clients, which is built on top of HTTP and a simple text format, providing a lightweight way to push servers, often referred to as “Event Stream”. It establishes a persistent connection between the client and the server, and pushes messages between the server and the client in real time through this connection.
Basic features of SSE:
- The protocol in HTML5 is a simple protocol based on plain text;
- An EventSource object that can be used by JavaScript on the browser side
EventSource provides three standard events, and supports disconnection and reconnection by default
eventdescriptiononopenOccurs when a connection to the server is successfully establishedonmessageOccurs when a message is received from the serveronerrorOccurs when there is an error
The data to be transferred has formatting requirements and must be [data:…\n… \n] or [retry:10\n]
SSE and WebSocket
When it comes to SSE, WebSocket is a natural mention. WebSocket is a kind ofHTML5The full-duplex communication protocol provided (refers to a communication protocol that allows two devices to send and receive data in both directions at the same time), based on:TCPprotocol, and multiplex HTTP handshake channels (allowing multiple HTTP requests and responses to be transmitted in a single TCP connection), commonly used for real-time communication between browsers and servers. Although SSE and WebSocket are similar in that they are both technologies used to push data from the server to the client in real time, there are certain differences:
1.SSE (Server-Sent Events)
- Simplicity: SSE uses a simple HTTP protocol, typically built on top of a standard HTTP or HTTPS connection. This makes it ideal for simple real-time notification scenarios, especially for a server pushing data to a client in one direction.
- Compatibility: SSE is compatible on the browser side because it is based on the standard HTTP protocol. Even in some environments that don’t support WebSockets, SSE can still be supported.
- Scope: SSE is suitable for one-way push notifications from the server to the client, such as real-time updates and event notifications. However, it only supports one-way communication from the server to the client, and the client cannot send messages directly to the server.
2.WebSocket
- Full-duplex communication: WebSocket provides full-duplex communication, allowing bi-directional, real-time communication between the client and the server. This makes it suitable for applications that require bi-directional data exchange, such as online chat, real-time collaboration, etc.
- Low latency: WebSocket has relatively low communication overhead because it uses a single, persistent connection, unlike SSE, which requires new connections to be created continuously. This reduces the latency of communication.
- Scope of application: WebSocket is suitable for applications that require real-time two-way communication, especially for those scenarios that require low-latency, high-frequency message exchange.
3. SSE or WebSocket?
- Simple notification scenario: SSE is a simple and effective choice if you only need the server to push simple notifications, event updates, etc., to the client, and do not need the client to communicate with the server in both directions.
- Bidirectional communication scenarios: If your application needs to implement real-time two-way communication, such as online chat, collaborative editing, etc., then WebSocket is a more suitable choice.
- Compatibility considerations: SSE may be a more viable option if your app may run in environments that don’t support WebSockets, or if you need to consider broader browser compatibility.
2. Code engineering
Experimental goal: to realize ChatGPT streaming interaction
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>springboot-demo</artifactId>
<groupId>com.et</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>sse</artifactId>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- java基础工具包 -->
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.8.9</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
</project>controller
package com.et.sse.controller;
import cn.hutool.core.util.IdUtil;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;
import java.io.IOException;
import java.util.Date;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@Controller
@RequestMapping("/chat")
public class ChatController {
Map<String, String> msgMap = new ConcurrentHashMap<>();
/**
* send meaaage
* @param msg
* @return
*/
@ResponseBody
@PostMapping("/sendMsg")
public String sendMsg(String msg) {
String msgId = IdUtil.simpleUUID();
msgMap.put(msgId, msg);
return msgId;
}
/**
* conversation
* @param msgId mapper with sendmsg
* @return
*/
@GetMapping(value = "/conversation/{msgId}", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public SseEmitter conversation(@PathVariable("msgId") String msgId) {
SseEmitter emitter = new SseEmitter();
String msg = msgMap.remove(msgId);
//mock chatgpt response
new Thread(() -> {
try {
for (int i = 0; i < 10; i++) {
ChatMessage chatMessage = new ChatMessage("test", new String(i+""));
emitter.send(chatMessage);
Thread.sleep(1000);
}
emitter.send(SseEmitter.event().name("stop").data(""));
emitter.complete(); // close connection
} catch (IOException | InterruptedException e) {
emitter.completeWithError(e); // error finish
}
}).start();
return emitter;
}
}chat.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>ChatGpt test</title>
<link rel="stylesheet" href="lib/element-ui/index.css">
<style type="text/css">
body{
background-color:white;
}
#outputCard{
height: 300px;
overflow:auto;
}
#inputCard{
height: 100px;
overflow:auto;
}
#outputBody{
line-height:30px;
}
.cursor-img{
height:24px;
vertical-align: text-bottom;
}
</style>
<script src="lib/jquery/jquery-3.6.0.min.js"></script>
<script src="lib/vue/vue.min.js"></script>
<script src="lib/element-ui/index.js"></script>
</head>
<body>
<h1 align="center">ChatGpt Test</h1>
<div id="chatWindow">
<el-row id="outputArea">
<el-card id="inputCard">
<div id="inputTxt">
</div>
</el-card>
<el-card id="outputCard">
<div id="outputBody">
<span id="outputTxt"></span>
<img v-if="blink" class="cursor-img" src="img/cursor-text-blink.gif" v-show="cursorImgVisible">
<img v-if="!blink" class="cursor-img" src="img/cursor-text-black.png" v-show="cursorImgVisible">
</div>
</el-card>
</el-row>
<el-row id="inputArea">
<el-col :span="21">
<el-input id="sendTxt" v-model="input" placeholder="input content" @keyup.native="keyUp"></el-input>
</el-col>
<el-col :span="3">
<el-button id="sendBtn" type="primary" :disabled="sendBtnDisabled" @click="sendMsg">send</el-button>
</el-col>
</el-row>
</div>
</body>
<script type="text/javascript">
var app = new Vue({
el: '#chatWindow',
data: {
input: '',
sendBtnDisabled: false,
cursorImgVisible: false,
blink: true
},
mounted: function(){
},
methods: {
keyUp: function(event){
if(event.keyCode==13){
this.sendMsg();
}
},
sendMsg: function(){
var that = this;
//init
$('#outputTxt').html('');
var sendTxt = $('#sendTxt').val();
$('#inputTxt').html(sendTxt);
$('#sendTxt').val('');
that.sendBtnDisabled = true;
that.cursorImgVisible = true;
//send request
$.ajax({
type: "post",
url:"/chat/sendMsg",
data:{
msg: sendTxt
},
contentType: 'application/x-www-form-urlencoded',
success:function(data){
var eventSource = new EventSource('/chat/conversation/'+data)
eventSource.addEventListener('open', function(e) {
console.log("EventSource connect success");
});
var blinkTimeout = null;
eventSource.addEventListener("message", function(evt){
var data = evt.data;
var json = JSON.parse(data);
var content = json.content ? json.content : '';
content = content.replaceAll('\n','<br/>');
console.log(json)
var outputTxt = $('#outputTxt');
outputTxt.html(outputTxt.html()+content);
var outputCard = $('#outputCard');
var scrollHeight = outputCard[0].scrollHeight;
outputCard.scrollTop(scrollHeight);
//cusor blink
that.blink = false;
window.clearTimeout(blinkTimeout);
//200ms blink=true
blinkTimeout = window.setTimeout(function(){
that.blink = true;
}, 200)
});
eventSource.addEventListener('error', function (e) {
console.log("EventSource error");
if (e.target.readyState === EventSource.CLOSED) {
console.log('Disconnected');
} else if (e.target.readyState === EventSource.CONNECTING) {
console.log('Connecting...');
}
});
eventSource.addEventListener('stop', e => {
console.log('EventSource connect end');
eventSource.close();
that.sendBtnDisabled = false;
that.cursorImgVisible = false;
}, false);
},
error: function(){
that.sendBtnDisabled = false;
that.cursorImgVisible = false;
}
});
}
}
})
</script>
</html>The above are just some of the key codes, all of which can be found in the repositories below
Code repositories
3. Testing
Start the Spring Boot application
Test streaming interactions
Visit the http://127.0.0.1:8088/chat.html and the effect is as follows
