Parsing Lines: Splitting Text and Converting Fields

Start with the problem

Text files arrive as lines. Each line holds several values separated by a delimiter — commas for CSV, tabs for TSV, spaces for simple logs. Your program needs those values as real types: int, double, std::string, not as one big string. The job is three steps: split, convert, validate. If any step fails, the whole line is suspect and should be rejected or logged, not silently corrupted.

How the idea works

The trace above shows a minimal parser. The line lives in a std::string_view, which is a non-owning reference to the original text — no copy. A position index pos tracks where the next field starts. Each parse_* function reads from pos until the delimiter or end, converts the slice, advances pos past the delimiter, and returns the typed value.

Conversion uses std::from_chars (C++17). It takes a pointer range and an output reference, returns a struct with ptr (one past the last consumed character) and ec (error code). No exceptions, no locale dependence, no heap. If ec == std::errc{}, the conversion succeeded and ptr - first tells you how many characters were used. That is exactly what pos needs to advance.

#include <charconv>
#include <string_view>
#include <optional>
 
bool parse_int(std::string_view sv, size_t& pos, char delim, int& out) {
    size_t end = sv.find(delim, pos);
    std::string_view field = sv.substr(pos, end - pos);
    auto [ptr, ec] = std::from_chars(field.data(), field.data() + field.size(), out);
    if (ec != std::errc{} || ptr != field.data() + field.size()) return false;
    pos = (end == std::string_view::npos) ? sv.size() : end + 1;
    return true;
}

parse_double is identical except the output type. parse_string copies the slice — no conversion, so it cannot fail unless you decide empty fields are invalid.

Why it is correct

string_view + from_chars avoids the classic pitfalls:

• No copy for the line or fields (the view refers to the original buffer).
• No exceptions from parsing — from_chars reports failure through ec.
• Exact consumption — the ptr == end check guarantees the whole field was a valid number, not "123abc" silently parsed as 123.
• Delimiter handling — advancing pos past the comma makes the next field start cleanly.

If a field fails, the parser returns false immediately. The caller decides: skip the line, log it, or stop. No partially parsed garbage leaks into your data structures.

What it costs

from_chars is typically faster than stoi/stod because it avoids locale, exceptions, and allocations. The whole parser runs in one pass over the line: O(n) characters, O(1) extra memory. For a contact book with thousands of lines, the time is dominated by file I/O, not parsing.

Pitfalls and variations

Whitespace. from_chars does not skip leading or trailing spaces. " 42 " fails. Trim the field first if your format allows spaces.

Quoted fields. Real CSV allows "hello, world" where the comma is inside quotes. A production parser tracks quote state. The simple delimiter scan above treats it as two fields — fine for controlled input, not for arbitrary CSV.

Missing fields. If a line ends early, find returns npos, substr takes the rest, and parse_* runs on a shorter slice. The conversion will fail if the field is empty or malformed, which is the correct signal.

Unicode. from_chars works on ASCII digits. For non-ASCII digit scripts, you need ICU or a wider parser. Most log/CSV files use ASCII digits.

Where it is used

The contact-book capstone (next lesson) reads one line per contact, splits on commas, and builds a Contact object. The same pattern appears in log processors, config loaders, and any tool that ingests structured text. The next lesson assembles the full program: multi-file build, structs, file I/O, and this parser together.

A complete parser example

Here is a self-contained parser you can drop into a project. It handles the three common types, returns std::optional to force the caller to check, and logs the line number on failure.

#include <charconv>
#include <fstream>
#include <iostream>
#include <optional>
#include <string>
#include <string_view>
#include <vector>
 
struct Record {
    int id;
    std::string name;
    double value;
};
 
std::optional<Record> parse_record(std::string_view line) {
    size_t pos = 0;
    Record rec;
 
    auto take_field = [&](char delim) -> std::string_view {
        size_t end = line.find(delim, pos);
        std::string_view field = line.substr(pos, end - pos);
        pos = (end == std::string_view::npos) ? line.size() : end + 1;
        return field;
    };
 
    // id
    std::string_view f_id = take_field(',');
    auto [p1, ec1] = std::from_chars(f_id.data(), f_id.data() + f_id.size(), rec.id);
    if (ec1 != std::errc{} || p1 != f_id.data() + f_id.size()) return std::nullopt;
 
    // name
    rec.name = std::string(take_field(','));
 
    // value
    std::string_view f_val = take_field(',');
    auto [p2, ec2] = std::from_chars(f_val.data(), f_val.data() + f_val.size(), rec.value);
    if (ec2 != std::errc{} || p2 != f_val.data() + f_val.size()) return std::nullopt;
 
    return rec;
}
 
int main(int argc, char* argv[]) {
    if (argc != 2) {
        std::cerr << "usage: " << argv[0] << " <csv-file>\n";
        return 1;
    }
    std::ifstream in(argv[1]);
    std::string line;
    size_t lineno = 0;
    std::vector<Record> records;
    while (std::getline(in, line)) {
        ++lineno;
        if (line.empty()) continue;
        if (auto rec = parse_record(line)) {
            records.push_back(*rec);
        } else {
            std::cerr << "line " << lineno << ": parse failed -> " << line << '\n';
        }
    }
    std::cout << "Parsed " << records.size() << " records.\n";
    for (const auto& r : records) {
        std::cout << r.id << ' ' << r.name << ' ' << r.value << '\n';
    }
}

Compile with g++ -std=c++17 -O2 the example -o parser and run ./parser contacts.csv. The program prints every valid record and reports malformed lines by number. This is the same pattern the capstone uses, with explicit error reporting added.

Performance and alternatives

std::from_chars is the fastest standard way to parse numbers in C++17. Benchmarks show it 2–5× faster than std::stoi/std::stod because it avoids locale lookup, exception machinery, and dynamic allocation. For a file with millions of lines, the difference is measurable: parsing 10 million integers takes roughly 0.3 seconds with from_chars versus 1.2 seconds with stoi on a typical laptop.

If you need even more speed, consider these options:

• SIMD parsing — libraries like simdjson or rapidcsv use vector instructions to parse multiple fields in parallel. They are harder to integrate but can reach 1–2 GB/s on modern CPUs.
• Memory-mapped I/O — mmap the file and parse directly from the mapped region. This avoids the read/getline syscall overhead and lets the OS page in data on demand. Combine with string_view over the mapped memory for zero-copy parsing.
• Custom number parser — for a known fixed format (e.g., exactly 10 digits), a hand-written loop that accumulates val = val * 10 + (c - '0') can beat from_chars by a small margin because it skips the generic checks.

For most applications, from_chars + string_view + getline is the sweet spot: standard, fast, zero-copy, and safe. Reach for the advanced options only when profiling proves the parser is the bottleneck.

Testing the parser

A parser is only as good as its test coverage. Test these cases explicitly:

// valid cases
assert(parse_record("1,hello,3.14") == Record{1, "hello", 3.14});
assert(parse_record("42,,0") == Record{42, "", 0.0});        // empty name
assert(parse_record("0,x,-2.5") == Record{0, "x", -2.5});   // negative double
 
// invalid cases
assert(!parse_record("abc,hello,1"));    // bad id
assert(!parse_record("1,hello,abc"));    // bad value
assert(!parse_record("1,hello"));        // missing field
assert(!parse_record(""));               // empty line
assert(!parse_record("1,hello,3.14,extra")); // extra field (ignored by current parser)

The last case reveals a design choice: the parser stops at the third comma and ignores trailing data. A stricter parser would check pos == line.size() after the last field and reject lines with extra columns. Choose the policy that matches your data contract.