Network State Recoder

Network State Recorder (NSR) - User Operations Manual

1. Features Overview

• Periodic Data Collection: Automatically fetches flow and graph data from NDTwin.
• Efficient Storage: Compresses JSON data into ZIP archives.
• Multi-threaded: Concurrent fetching, writing, and compression.
• Graceful Shutdown: Handles SIGINT/SIGTERM for clean data saving.

2. Usage Guide

Starting NSR

Option 1: Background Mode (Recommended)

Use this for long-term data collection. It runs NSR in the background using nohup.

./start_network_state_recorder.sh

*Note: this script will change the value of display_on_console in config file to false

Option 2: Foreground Mode (Debugging)

In order to monitor real-time logs in the terminal, you need to set the display_on_console to true

python3 network_state_recorder.py

Checking Status

To verify if NSR is currently running:

pgrep -f network_state_recorder.py

If a process ID (PID) is returned, NSR is running.

Stopping NSR

Option 1: Using the Stop Script

./stop_network_state_recorder.sh

*Note: If permissions deny execution, try: sudo ./stop_network_state_recorder.sh* *Note: this script will change the value of display_on_console in config file to true

Option 2: Manual Termination

• Foreground: Press Ctrl+C.
• Background:

sudo kill -15 $(pgrep -f network_state_recorder.py)

3. Output Data & Logs

Data Location & Format

All data is stored in the ./recorded_info/ directory.

Naming Convention:

• Raw: YYYY_MM_DD_HH-MM-SS_<datatype>.json
• Compressed: YYYY_MM_DD_HH-MM-SS_<datatype>_json.zip

Data Types:

• flowinfo: Network flow information.
• graphinfo: Network graph/topology information.

JSON Structure

Each file contains newline-delimited JSON records.

Common Header:

{"timestamp": 1704067200000, ...}

Flow Info Format (flowinfo):

{"timestamp": 1704067200000, "flowinfo":{[...]}}

Graph Info Format (graphinfo):

{"timestamp": 1704067200000, "edges":{[...]}, "nodes":[{...}]}

Logging

NSR logs are immediately written to the ./logs/ folder (not displayed in the terminal during background execution).

Log File Format: logs/NSR_YYYY-MM-DD.log

Real-time Monitoring Background NSR:

tail -f logs/NSR_$(date +%Y-%m-%d).log

4. Troubleshooting