Configure Grafana for Colored JSON Logs

To configure Grafana to display logs in the same format as in the terminal with color-coded levels, you need to properly configure JSON log processing and ensure that log levels are correctly recognized. The main problem for most beginners is that Grafana cannot automatically detect log levels if they are not properly extracted from JSON.

Table of Contents

• Basic Settings for Displaying JSON Logs
• Configuring Promtail for Proper Log Level Processing
• Pino-Loki Configuration
• Configuring the Log Panel in Grafana
• Troubleshooting Common Problems
• Examples for Node.js Applications

Basic Settings for Displaying JSON Logs

For Grafana to correctly display JSON logs with color-coded level indicators, you need to perform several key steps:

1. Log formatting - your logs should be in JSON format with a level field, for example:

   json

   {"level": "info", "time": 1634567890, "pid": 1234, "hostname": "server", "msg": "Application started"}
   

2. Log level extraction - Grafana should recognize the level field as level for automatic coloring.

3. Proper data source configuration - appropriate parsers should be used in Loki.

Important: As noted in the Grafana documentation, if your logs contain a level field, Grafana automatically uses its values to determine the color of each line.

Configuring Promtail for Proper Log Level Processing

The most common mistake for beginners is incorrect Promtail configuration. Your configuration should include:

scrape_configs:
- job_name: pino-logs
  pipeline_stages:
    # Main transformation for pino logs
    - json:
        expressions:
          level: level
          msg: msg
          time: time
    
    # Convert timestamp to readable format
    - timestamp:
        source: time
        format: UnixMs
    
    # Extract level for color display
    - labels:
        level:

Tip: If your logs have a non-standard structure, use examples from the Promtail documentation to configure parsing.

Pino-Loki Configuration

Your Node.js application should use the correct pino-loki configuration:

const pino = require('pino');
const LokiTransport = require('pino-loki');

const logger = pino({
  level: 'info',
  formatters: {
    level: (label) => {
      return { level: label };
    }
  }
}, LokiTransport({
  host: 'http://loki:3100',
  labels: { job: 'pino-logs' },
  interval: 5
}));

Important: As noted in the official pino documentation, when using pino-loki, ensure that log levels are properly formatted for correct Grafana processing.

Configuring the Log Panel in Grafana

To create a panel with colored JSON logs:

1. Create a new panel of type “Logs”
2. In the query settings, select Loki as the data source
3. Use the following query:
   {job="pino-logs"} |= ""
   

4. In the panel settings, go to the “Panel options” tab and ensure that “Enable log level colors” is enabled

For better JSON display, use the “Logs Drilldown JSON viewer” mode, as described in the Grafana documentation:

Note: The “Logs Drilldown JSON viewer” mode allows you to easily view and interact with formatted JSON logs, filter them, and make your visualizations more relevant.

Troubleshooting Common Problems

Problem: Logs are not colored by levels

Solution: Ensure that:

• The level field is named level
• Promtail correctly extracts this field
• Color coding is enabled in the panel settings

Problem: JSON is displayed as a table

Solution: In the panel settings, select the “Logs Drilldown JSON viewer” display format instead of the standard table view.

Problem: No access to Grafana Playground

Solution: Configuration examples can be found in the Grafana dashboards repository, which demonstrates using Loki for visualizing Pino logs.

Examples for Node.js Applications

Here is a complete setup example:

docker-compose.yml:

version: '3.8'
services:
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    
  promtail:
    image: grafana/promtail:latest
    volumes:
      - ./promtail-config.yaml:/etc/promtail/config.yml
      - /var/log:/var/log
    command: -config.file=/etc/promtail/config.yml
    
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    depends_on:
      - loki

promtail-config.yaml:

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
- job_name: pino-logs
  pipeline_stages:
    - json:
        expressions:
          level: level
          msg: msg
          time: time
    - timestamp:
        source: time
        format: UnixMs
    - labels:
        level:

Tip: As shown in this Node.js log setup guide, proper configuration of the log processing pipeline is key to successful visualization.

Conclusion

1. Proper JSON structure - your logs should contain a level field for automatic coloring in Grafana.

2. Promtail configuration - use pipelines to extract log levels and convert timestamp formats.

3. Pino-loki configuration - ensure that log levels are properly formatted and sent to Loki.

4. Panel configuration - use the “Logs Drilldown JSON viewer” mode for convenient viewing of JSON logs with level color indication.

5. Problem debugging - if colors are not displaying, check Promtail settings and ensure that the level field is properly extracted.

By following these steps, you can configure Grafana to display logs in the same format as in the terminal, with level color indication and convenient JSON formatting.

Sources

1. Structured Logging in the Grafana monitoring stack - Medium
2. Pino HTTP Logs - Grafana Labs
3. Logs visualization in Grafana - Official Documentation
4. Logs in Explore - Grafana Documentation
5. Logs Drilldown JSON viewer - Grafana Documentation
6. pino Loki transport configuration
7. Visualizing Loki JSON logs in Grafana - Stack Overflow
8. Ultimate Guide: Shipping Node.js Application Logs to Loki