Project: Command-Line Hex Dumper (xxd Clone) · Pointers, Memory, and Arrays

Start with the problem

Inspecting raw binary files—compiled ELF binaries, core dumps, raw network packets, database write-ahead logs, and encrypted payloads—is an everyday task in systems programming. Unlike text files, binary data cannot be displayed directly in a text editor or emitted straight to the terminal using cat. Non-printable bytes and control codes (such as 0x00, 0x1B, and 0x07) can trigger escape codes, silence terminal output, or scramble cursor positions.

The standard tool for inspecting arbitrary binary streams is xxd (or hexdump -C). It formats raw bytes into three distinct columns:

text

00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000  .ELF............
00000010: 0300 3e00 0100 0000 5011 0000 0000 0000  ..>.....P.......

Memory Offset: An 8-digit hexadecimal counter tracking the byte position in the stream.
Hexadecimal Octets: The raw values of the bytes displayed in hexadecimal, grouped for readability (e.g., in 2-byte groups).
ASCII Sidebar: A human-readable representation where printable ASCII characters (0x20–0x7E) are displayed verbatim and non-printable bytes are replaced with a placeholder dot (.).

Building this tool in pure C tests the foundational concepts of systems programming:

Command-line flag parsing: Handling POSIX flags (-c, -g, -s, -l, -u) and file arguments without third-party libraries.
Stream I/O: Reading cleanly from either standard input (stdin) or regular files using buffered streams.
Pointer arithmetic: Stepping through memory buffers using pointers, calculating offsets, and avoiding array index bounds bugs.
Data representation: Managing unsigned byte types (uint8_t) to prevent unintended sign extension during integer promotion.

How the idea works

The application follows a modular, streaming pipeline:

text

                             Execution Flow
┌────────────────────────────────────────────────────────────────────────┐
│ 1. CLI Flag Parsing (getopt)                                           │
│    - Parse -c (cols), -g (grouping), -s (seek), -l (limit), -u (upper) │
│    - Open target file or bind to stdin                                 │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
┌───────────────────────────────────▼────────────────────────────────────┐
│ 2. Seek & Limit Initialization                                         │
│    - If -s specified: fseeko() to initial offset                       │
│    - Track bytes_remaining if -l is configured                         │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
┌───────────────────────────────────▼────────────────────────────────────┐
│ 3. Chunk Streaming Loop (fread)                                        │
│    - Read up to 'cols' bytes into stack-allocated uint8_t buffer       │
│    - Stop on EOF or when -l byte limit is satisfied                    │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
┌───────────────────────────────────▼────────────────────────────────────┐
│ 4. Line Formatter (Pointer Stepping Engine)                            │
│    - [Offset]:  snprintf("%08zx: ")                                    │
│    - [Hex]:     Walk pointers, emit %02x, insert grouping spaces       │
│    - [Padding]: Calculate missing byte positions for partial lines     │
│    - [ASCII]:   Walk pointers, evaluate isprint(*p), print sidebar     │
└────────────────────────────────────────────────────────────────────────┘

1. Pointer Stepping Across the Buffer

Instead of relying on multi-index arithmetic, we traverse the read buffer using a pair of pointer anchors:

const uint8_t *curr = buffer;
const uint8_t *end  = buffer + bytes_read;
 
while (curr < end) {
    uint8_t byte = *curr++;
    /* Process byte */
}

This ensures linear access through the buffer, matching how the CPU's prefetcher loads data into L1 cache lines.

2. Handling the Last Line (Partial Reads)

When a file's total size is not an exact multiple of the column width (e.g., 23 bytes with a 16-column layout), the second line will contain only 7 bytes:

text

00000000: 4865 6c6c 6f20 576f 726c 6421 0a43 2050  Hello World!.C P
00000010: 726f 6772 616d                            rogram

If we do not account for the 9 missing bytes in line 2, the ASCII sidebar (rogram) will shift immediately adjacent to the hex output, corrupting vertical column alignment.

To prevent this:

Calculate the number of missing bytes: missing = cols - bytes_read.
Compute how many spaces those missing bytes would have occupied (including group separation spaces).
Emit padding spaces to push the ASCII sidebar into its fixed column.

Complete Implementation: `xxd_clone.c`

Here is the complete, self-contained implementation in standard C (C99/POSIX.1-2008):

#define _POSIX_C_SOURCE 200809L
 
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <stdbool.h>
#include <string.h>
#include <ctype.h>
#include <unistd.h>
#include <errno.h>
 
#define DEFAULT_COLS 16
#define DEFAULT_GROUP 2
#define MAX_COLS 256
 
typedef struct {
    size_t cols;          /* Octets per line (-c) */
    size_t group_bytes;   /* Group size in octets (-g) */
    off_t seek_offset;    /* Starting file offset (-s) */
    int64_t length_limit; /* Maximum bytes to dump (-l, -1 for unlimited) */
    bool uppercase;       /* Uppercase hex output (-u) */
    const char *filepath; /* Input filepath (NULL for stdin) */
} Config;
 
static void print_usage(const char *prog_name) {
    fprintf(stderr,
        "Usage: %s [options] [infile]\n"
        "Options:\n"
        "  -c <cols>     Format <cols> octets per line (1-%d, default: %d)\n"
        "  -g <bytes>    Group output every <bytes> octets (default: %d, 0 to disable)\n"
        "  -s <offset>   Seek to <offset> before dumping (prefix 0x for hex)\n"
        "  -l <len>      Stop after reading <len> octets\n"
        "  -u            Use uppercase hex letters (default: lowercase)\n"
        "  -h            Display this help and exit\n",
        prog_name, MAX_COLS, DEFAULT_COLS, DEFAULT_GROUP);
}
 
static bool parse_args(int argc, char **argv, Config *cfg) {
    cfg->cols = DEFAULT_COLS;
    cfg->group_bytes = DEFAULT_GROUP;
    cfg->seek_offset = 0;
    cfg->length_limit = -1;
    cfg->uppercase = false;
    cfg->filepath = NULL;
 
    int opt;
    while ((opt = getopt(argc, argv, "c:g:s:l:uh")) != -1) {
        char *endptr = NULL;
        switch (opt) {
            case 'c': {
                long val = strtol(optarg, &endptr, 10);
                if (*endptr != '\0' || val < 1 || val > MAX_COLS) {
                    fprintf(stderr, "Error: Invalid column count '%s' (must be 1-%d).\n",
                            optarg, MAX_COLS);
                    return false;
                }
                cfg->cols = (size_t)val;
                break;
            }
            case 'g': {
                long val = strtol(optarg, &endptr, 10);
                if (*endptr != '\0' || val < 0 || val > MAX_COLS) {
                    fprintf(stderr, "Error: Invalid group size '%s'.\n", optarg);
                    return false;
                }
                cfg->group_bytes = (size_t)val;
                break;
            }
            case 's': {
                /* Supports decimal or hex (e.g., 0x100) */
                int base = (strncmp(optarg, "0x", 2) == 0 || strncmp(optarg, "0X", 2) == 0) ? 16 : 10;
                long long val = strtoll(optarg, &endptr, base);
                if (*endptr != '\0' || val < 0) {
                    fprintf(stderr, "Error: Invalid seek offset '%s'.\n", optarg);
                    return false;
                }
                cfg->seek_offset = (off_t)val;
                break;
            }
            case 'l': {
                long long val = strtoll(optarg, &endptr, 10);
                if (*endptr != '\0' || val < 0) {
                    fprintf(stderr, "Error: Invalid length limit '%s'.\n", optarg);
                    return false;
                }
                cfg->length_limit = (int64_t)val;
                break;
            }
            case 'u':
                cfg->uppercase = true;
                break;
            case 'h':
                print_usage(argv[0]);
                exit(EXIT_SUCCESS);
            default:
                print_usage(argv[0]);
                return false;
        }
    }
 
    if (optind < argc) {
        cfg->filepath = argv[optind];
    }
    return true;
}
 
/* Renders a single line of canonical hex + ASCII output */
static void dump_line(const uint8_t *buf, size_t count, size_t offset, const Config *cfg) {
    const char *hex_fmt = cfg->uppercase ? "%02X" : "%02x";
 
    /* 1. Print 8-digit hexadecimal memory offset */
    printf("%08zx: ", offset);
 
    /* 2. Print hexadecimal octets with grouping */
    size_t chars_printed = 0;
    for (size_t i = 0; i < cfg->cols; i++) {
        if (i < count) {
            printf(hex_fmt, buf[i]);
            chars_printed += 2;
        } else {
            /* Missing byte padding */
            printf("  ");
            chars_printed += 2;
        }
 
        /* Check if grouping space is required */
        if (cfg->group_bytes > 0 && ((i + 1) % cfg->group_bytes == 0) && (i + 1 < cfg->cols)) {
            putchar(' ');
            chars_printed += 1;
        }
    }
 
    /* Trailing spacing before ASCII sidebar */
    printf("  ");
 
    /* 3. Print printable ASCII sidebar */
    const uint8_t *curr = buf;
    const uint8_t *end = buf + count;
 
    while (curr < end) {
        uint8_t ch = *curr++;
        putchar(isprint(ch) ? (char)ch : '.');
    }
 
    putchar('\n');
}
 
int main(int argc, char **argv) {
    Config cfg;
    if (!parse_args(argc, argv, &cfg)) {
        return EXIT_FAILURE;
    }
 
    FILE *fp = stdin;
    if (cfg.filepath != NULL && strcmp(cfg.filepath, "-") != 0) {
        fp = fopen(cfg.filepath, "rb");
        if (!fp) {
            fprintf(stderr, "Error: Cannot open '%s': %s\n", cfg.filepath, strerror(errno));
            return EXIT_FAILURE;
        }
    }
 
    /* Apply seek offset if specified */
    if (cfg.seek_offset > 0) {
        if (fseeko(fp, cfg.seek_offset, SEEK_SET) != 0) {
            /* If input is non-seekable (e.g., stdin pipe), consume bytes manually */
            off_t skipped = 0;
            uint8_t discard[512];
            while (skipped < cfg.seek_offset) {
                size_t to_read = sizeof(discard);
                if ((off_t)to_read > (cfg.seek_offset - skipped)) {
                    to_read = (size_t)(cfg.seek_offset - skipped);
                }
                size_t n = fread(discard, 1, to_read, fp);
                if (n == 0) break;
                skipped += n;
            }
        }
    }
 
    /* Fixed-size stack buffer for reading chunks */
    uint8_t buffer[MAX_COLS];
    size_t current_offset = (size_t)cfg.seek_offset;
    int64_t total_emitted = 0;
 
    while (1) {
        /* Determine how many bytes to read in this cycle */
        size_t to_read = cfg.cols;
        if (cfg.length_limit >= 0) {
            int64_t remaining = cfg.length_limit - total_emitted;
            if (remaining <= 0) break;
            if ((int64_t)to_read > remaining) {
                to_read = (size_t)remaining;
            }
        }
 
        size_t bytes_read = fread(buffer, 1, to_read, fp);
        if (bytes_read == 0) {
            break; /* EOF or read error */
        }
 
        dump_line(buffer, bytes_read, current_offset, &cfg);
 
        current_offset += bytes_read;
        total_emitted += bytes_read;
 
        if (bytes_read < to_read) {
            break; /* Reached end of file early */
        }
    }
 
    if (fp != stdin) {
        fclose(fp);
    }
 
    return EXIT_SUCCESS;
}

Compiling and Running

Compile with strict compiler warnings:

bash

gcc -Wall -Wextra -Wpedantic -O2 xxd_clone.c -o xxd_clone

1. Hex Dumper on a Compiled Binary

bash

./xxd_clone -c 16 -g 2 -l 32 xxd_clone

Output:

text

00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000  .ELF............
00000010: 0300 3e00 0100 0000 7010 0000 0000 0000  ..>.....p.......

2. Reading from Stdin via Pipe

bash

echo "The Runtime Theory Systems Programming" | ./xxd_clone -c 8 -g 1 -u

Output:

text

00000000: 54 68 65 20 52 75 6E 74  The Runt
00000008: 69 6D 65 20 54 68 65 6F  ime Theo
00000010: 72 79 20 53 79 73 74 65  ry Syste
00000018: 6D 73 20 50 72 6F 67 72  ms Progr
00000020: 61 6D 6D 69 6E 67 0A     amming.

3. Seeking into a Binary File

bash

./xxd_clone -s 0x10 -l 16 xxd_clone

Output:

text

00000010: 0300 3e00 0100 0000 7010 0000 0000 0000  ..>.....p.......

Why it is correct

Understanding byte mechanics explains why this implementation avoids common bugs found in naive implementations:

1. Preventing Sign Extension Bugs

In standard C, passing a signed char to printf("%02x", ch) triggers default argument promotion to int. If the byte has its most significant bit set (e.g., 0x89), it is promoted to 0xFFFFFF89 on 32-bit/64-bit systems.

By typing the buffer as uint8_t (defined in <stdint.h>), the value is treated as an unsigned integer (0x00000089), guaranteeing clean, two-digit hex output.

2. Dual-Mode Seeking Strategy

Standard disk files support fseeko() for instant O(1) random access. However, command-line utilities frequently operate on standard input pipes (e.g., cat file | ./xxd_clone -s 1024). Pipes are non-seekable streams where fseeko() returns ESPIPE.

The implementation addresses this gracefully:

if (fseeko(fp, cfg.seek_offset, SEEK_SET) != 0) {
    /* Fall back to streaming byte discard for non-seekable pipes */
    while (skipped < cfg.seek_offset) { ... }
}

This guarantees identical seek behavior across regular files and piped streams.

3. Constant Stack-Only Footprint (O(1) Memory)

Regardless of whether the input file is 10 kilobytes or 50 gigabytes, the program never calls malloc(). The active stream buffer is allocated on the stack (uint8_t buffer[MAX_COLS]), consuming fewer than 512 bytes of total memory.

What it costs

Operation	Latency / Complexity	Primary Resource
Command-line argument parse	~1–2 micros	CPU instruction execution
Seek operation (`fseeko`)	~5–10 micros	Kernel file table offset update
16-byte chunk read (`fread`)	~50–150 ns	C standard library stream buffer copy
Formatted line output (`printf`)	~300–600 ns	User-space string assembly and stdout buffering
Memory consumption	< 4 KB total	Resident stack memory; 0 bytes dynamic heap

Pitfalls and variations

Pitfall 1: Unchecked Dynamic Format Specifiers

Never construct format strings dynamically via concatenation with untrusted input:

/* DANGEROUS FORMAT STRING BUG */
char fmt[32];
sprintf(fmt, "%%%ds", user_width); /* Vulnerable to format injection */

Instead, select between static format string literals (e.g., cfg->uppercase ? "%02X" : "%02x") or use precision specifiers (printf("%.*s", len, str)).

Pitfall 2: Forgetting Binary Mode on Windows (`"rb"`)

Opening a file with fopen(filepath, "r") treats the file as text mode. On Windows platforms:

Any occurrence of byte 0x1A is interpreted as an EOF marker, prematurely halting output.
Byte sequence 0x0D 0x0A (\r\n) is translated to 0x0A (\n), altering the byte offsets of all subsequent data.
The Rule: Binary file inspection tools must always pass "rb" to fopen().

Where it is used

Hexadecimal dump engines are foundational across systems programming:

GDB / LLDB: The x/x memory inspection command executes identical pointer stepping and byte formatting to render stack and heap memory.
Wireshark / tcpdump: Network packet payload inspection panes format raw Ethernet frames and TCP segments using canonical hex and ASCII sidebars.
Git: The internal object storage engine inspects object headers and raw SHA blobs using hex dump streams.

With this Capstone complete, you have mastered the Foundations Tier: pointer mechanics, binary layouts, memory inspection, and stream I/O in C.

Start with the problem

How the idea works

1. Pointer Stepping Across the Buffer

2. Handling the Last Line (Partial Reads)

Complete Implementation: xxd_clone.c

Compiling and Running

1. Hex Dumper on a Compiled Binary

2. Reading from Stdin via Pipe

3. Seeking into a Binary File

Why it is correct

1. Preventing Sign Extension Bugs

2. Dual-Mode Seeking Strategy

3. Constant Stack-Only Footprint (O(1) Memory)

What it costs

Pitfalls and variations

Pitfall 1: Unchecked Dynamic Format Specifiers

Pitfall 2: Forgetting Binary Mode on Windows ("rb")

Where it is used

Complete Implementation: `xxd_clone.c`

Pitfall 2: Forgetting Binary Mode on Windows (`"rb"`)