Springboot integrated graylog quick start demo

一、graylog introduce

GrayLog is a lightweight distributed log management platform and an open source log aggregation, analysis, auditing, display and early warning tool. Functionally, it is similar to ELK, but much simpler and lighter than ELK. Relying on the advantages of being more concise, efficient, and easy to deploy and use, it has quickly become favored by many companies. GrayLog contains three modules: Elasticsearch, MongoDb and Graylog

二、graylog environment setup

Docker-compose is used here to build the graylog test environment. The specific configuration is as follows

# docker-compose：https://docs.graylog.org/en/3.3/pages/installation/docker.html
version: '3'
services:
  mongo:
    image: mongo:3
    container_name: graylog_demo_mongo
    restart: unless-stopped
    #    volumes:
    #      - ./graylog/mongo_data:/data/db
    networks:
      - graylog
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.10
    container_name: graylog_demo_elasticsearch
    restart: unless-stopped
    #    volumes:
    #      - ./graylog/es_data:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    deploy:
      resources:
        limits:
          memory: 1g
    networks:
      - graylog
  graylog:
    image: graylog/graylog:3.3
    container_name: graylog_demo_graylog
    restart: unless-stopped
    #    volumes:
    #      - ./graylog/graylog_data_journal:/usr/share/graylog/data/journal
    environment:
      - GRAYLOG_PASSWORD_SECRET=somepasswordpepper # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 # Password: admin
      - GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/ # TODO here write the loacl ip
    ports:
      # Graylog web interface and REST API
      - 9000:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
    networks:
      - graylog
    depends_on:
      - mongo
      - elasticsearch
#bridge graylog -> communicate easyly
networks:
  graylog:
    driver: bridge

start service

docker-compose -f docker-compose-graylog.yml -p graylog_demo up -d

access url：http://ip:9001 default login name/password：admin/admin

2.1、new input http input

test send is right?

curl -XPOST http://localhost:12201/gelf -p0 -d '{"message":"hello这是一条消息", "host":"127.0.0.1", "facility":"test", "topic": "meme"}'

2.2、create gref udp input

because http receiver is not efficient，so we use UDP receiver to collect the log for springboot application

三、code project

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>springboot-demo</artifactId>
        <groupId>com.et</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
<artifactId>grayLog</artifactId>
    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
        </dependency>
        <dependency>
            <groupId>biz.paluch.logging</groupId>
            <artifactId>logstash-gelf</artifactId>
            <version>1.11.1</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
    </dependencies>
</project>

HelloWorldController.java

package com.et.graylog.controller;
import com.et.graylog.service.DemoService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import java.util.HashMap;
import java.util.Map;
@Slf4j
@Controller
public class HelloWorldController {
    @Autowired
    DemoService demoService;
    @RequestMapping("/hello")
    @ResponseBody
    public Map<String, Object> showHelloWorld(){
        Map<String, Object> map = new HashMap<>();
        map.put("msg", "HelloWorld");
        log.info("my name is HelloWorldController");
        demoService.hello();
        return map;
    }
    @GetMapping("log")
    public String log(){
        log.error("error");
        log.warn("warn");
        log.info("info");
        log.debug("debug");
        int i = 1/0;
        return "success";
    }
}

DemoApplication.java

package com.et.graylog;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class DemoApplication {
   public static void main(String[] args) {
      SpringApplication.run(DemoApplication.class, args);
   }
}

application.yaml

server:
  port: 8088
groylog:
  ip: 127.0.0.1
  port: 12201
  maximumMessageSize: 8192

logback.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="GELF" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:127.0.0.1</host><!--graylog server ip-->
        <port>12201</port><!--graylog udp port-->
        <version>1.1</version>
        <facility>springboot-with-graylog</facility>
        <extractStackTrace>true</extractStackTrace>
        <filterStackTrace>true</filterStackTrace>
        <mdcProfiling>true</mdcProfiling>
        <timestampPattern>yyyy-MM-dd HH:mm:ss,SSSS</timestampPattern>
        <maximumMessageSize>8192</maximumMessageSize>
        <!-- This are fields using MDC -->
        <mdcFields>mdcField1,mdcField2</mdcFields>
        <dynamicMdcFields>mdc.*,(mdc|MDC)fields</dynamicMdcFields>
        <includeFullMdc>true</includeFullMdc>
    </appender>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <root level="INFO">
        <appender-ref ref="GELF"/>
        <appender-ref ref="CONSOLE"/>
    </root>
</configuration>

code reponsitory

• https://github.com/Harries/springboot-demo

四、test

start springboot application

"C:\Program Files\Java\jdk1.8.0_251\bin\java.exe" -Dvisualvm.id=505582826333400 "-javaagent:D:\huiyida\develop\IntelliJ IDEA Community Edition 2022.1.3\lib\idea_rt.jar=57262:D:\huiyida\develop\IntelliJ IDEA Community Edition 2022.1.3\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_251\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\rt.jar;D:\IdeaProjects\ETFramework\grayLog\target\classes;D:\jar_repository\org\springframework\boot\spring-boot-starter-web\2.2.5.RELEASE\spring-boot-starter-web-2.2.5.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter\2.2.5.RELEASE\spring-boot-starter-2.2.5.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-logging\2.2.5.RELEASE\spring-boot-starter-logging-2.2.5.RELEASE.jar;D:\jar_repository\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;D:\jar_repository\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\jar_repository\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar;D:\jar_repository\org\apache\logging\log4j\log4j-to-slf4j\2.12.1\log4j-to-slf4j-2.12.1.jar;D:\jar_repository\org\apache\logging\log4j\log4j-api\2.12.1\log4j-api-2.12.1.jar;D:\jar_repository\org\slf4j\jul-to-slf4j\1.7.30\jul-to-slf4j-1.7.30.jar;D:\jar_repository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;D:\jar_repository\org\springframework\spring-core\5.2.4.RELEASE\spring-core-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-jcl\5.2.4.RELEASE\spring-jcl-5.2.4.RELEASE.jar;D:\jar_repository\org\yaml\snakeyaml\1.25\snakeyaml-1.25.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-json\2.2.5.RELEASE\spring-boot-starter-json-2.2.5.RELEASE.jar;D:\jar_repository\com\fasterxml\jackson\core\jackson-databind\2.10.2\jackson-databind-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\core\jackson-annotations\2.10.2\jackson-annotations-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\core\jackson-core\2.10.2\jackson-core-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.10.2\jackson-datatype-jdk8-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.10.2\jackson-datatype-jsr310-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.10.2\jackson-module-parameter-names-2.10.2.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-tomcat\2.2.5.RELEASE\spring-boot-starter-tomcat-2.2.5.RELEASE.jar;D:\jar_repository\org\apache\tomcat\embed\tomcat-embed-core\9.0.31\tomcat-embed-core-9.0.31.jar;D:\jar_repository\org\apache\tomcat\embed\tomcat-embed-el\9.0.31\tomcat-embed-el-9.0.31.jar;D:\jar_repository\org\apache\tomcat\embed\tomcat-embed-websocket\9.0.31\tomcat-embed-websocket-9.0.31.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-validation\2.2.5.RELEASE\spring-boot-starter-validation-2.2.5.RELEASE.jar;D:\jar_repository\jakarta\validation\jakarta.validation-api\2.0.2\jakarta.validation-api-2.0.2.jar;D:\jar_repository\org\hibernate\validator\hibernate-validator\6.0.18.Final\hibernate-validator-6.0.18.Final.jar;D:\jar_repository\org\jboss\logging\jboss-logging\3.4.1.Final\jboss-logging-3.4.1.Final.jar;D:\jar_repository\com\fasterxml\classmate\1.5.1\classmate-1.5.1.jar;D:\jar_repository\org\springframework\spring-web\5.2.4.RELEASE\spring-web-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-beans\5.2.4.RELEASE\spring-beans-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-webmvc\5.2.4.RELEASE\spring-webmvc-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-aop\5.2.4.RELEASE\spring-aop-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-context\5.2.4.RELEASE\spring-context-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-expression\5.2.4.RELEASE\spring-expression-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot-autoconfigure\2.2.5.RELEASE\spring-boot-autoconfigure-2.2.5.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot\2.2.5.RELEASE\spring-boot-2.2.5.RELEASE.jar;D:\jar_repository\biz\paluch\logging\logstash-gelf\1.11.1\logstash-gelf-1.11.1.jar;D:\jar_repository\org\projectlombok\lombok\1.18.12\lombok-1.18.12.jar" com.et.graylog.DemoApplication. ____ _ __ _ _
 /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/ ___)| |_)| | | | | || (_| | ) ) ) )
 ' |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot :: (v2.2.5.RELEASE)
10:07:22.906 INFO com.et.graylog.DemoApplication - Starting DemoApplication on BLHHUAPC with PID 30200 (D:\IdeaProjects\ETFramework\grayLog\target\classes started by Dell in D:\IdeaProjects\ETFramework)
10:07:22.918 INFO com.et.graylog.DemoApplication - No active profile set, falling back to default profiles: default
10:07:23.801 INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8088 (http)
10:07:23.811 INFO o.a.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8088"]
10:07:23.813 INFO o.a.catalina.core.StandardService - Starting service [Tomcat]
10:07:23.813 INFO o.a.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.31]
10:07:23.885 INFO o.a.c.c.C.[Tomcat].[localhost].[/] - Initializing Spring embedded WebApplicationContext
10:07:23.887 INFO o.s.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 930 ms
10:07:24.025 INFO o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService 'applicationTaskExecutor'
10:07:24.124 INFO o.a.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-8088"]
10:07:24.143 INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat started on port(s): 8088 (http) with context path ''
10:07:24.147 INFO com.et.graylog.DemoApplication - Started DemoApplication in 1.564 seconds (JVM running for 2.09)

login graylog，check whether the log has been sent?

五、renference

• http://blog.healerjean.com/docker/log/2018/03/16/2_Docker%E5%AE%89%E8%A3%85graylog%E5%92%8C%E4%BD%BF%E7%94%A8%E6%95%99%E7%A8%8B/
• https://archivedocs.graylog.org/en/3.3/