CPP/C++ Standard Library Input and Output
Table of Contents
- 1. C++ IO Input and Output System
- 1.1. Environment Variables for Program Configuration
- 1.2. C-legacy Printf-like IO functions - format specifiers and examples
- 1.3. C-legacy File IO functions
- 1.4. Standard Library IO Class Hierarchy
- 1.5. Standard Library IO Manipulators
- 1.6. Standard Library IO stream redirection
- 1.7. Reading data from generic Input Stream - read data from file, console and memory
- 1.8. Reading ELF binary files
- 1.9. Reading whole input stream to string
- 1.10. Reading whole file to string
- 1.11. IO - Insertion << and extraction >> operators
- 1.12. Custom IO manipulators
- 1.13. Redirect IO .rdbuf()
- 1.14. Saving and restoring IO flags
1 C++ IO Input and Output System
1.1 Environment Variables for Program Configuration
Environment variables are available in most operating systems such as Linux, MacOSx, BSD and Windows. They can be used for setting program configuration at initialization.
See:
- Environment variable - Wikipedia
- Environment Variables - Windows applications | Microsoft Docs
- How To Read and Set Environmental and Shell Variables on a Linux VPS | DigitalOcean
- What are environment variables in Bash? | Opensource.com
- Environment variables - ArchWiki
File: envvar.cpp
#include <iostream> #include <string> template<typename T> T get_env_value(std::string const& name, T default_value); // Template specialization for std::string template<> std::string get_env_value(std::string const& name, std::string default_value) { const char* p = std::getenv(name.c_str()); if(p == nullptr) { return default_value; } return p; } // Template specialization for size_t template<> size_t get_env_value(std::string const& name, size_t default_value) { const char* p = std::getenv(name.c_str()); if(p == nullptr) { return default_value; } return std::stoul(p); } // Template specialization for double template<> double get_env_value(std::string const& name, double default_value) { const char* p = std::getenv(name.c_str()); if(p == nullptr) { return default_value; } return std::stod(p); } int main(int argc, char* argv[]) { // Check whether environment variable "PROGRAM_USER_KEY" is set if(const char* env = std::getenv("PROGRAM_USER_KEY")) { std::cout << " [1] Environment vartiable set => PROGRAM_USER_KEY = " << env << std::endl; } else { std::cout <<" [1] Environment variable PROGRAM_USER_KEY not set." << std::endl; } auto storage_path = get_env_value<std::string>("PROGRAM_STORAGE_PATH" , "/home/dummy/.data"); auto number_of_threads = get_env_value<size_t>("PROGRAM_NTHREADS", 10); auto timeout = get_env_value<double>("PROGRAM_TIMEOUT", 1.0); std::cout << " ==== Program Runtime Configuration =======" << std::endl; std::cout << " ENV[PROGRAM_STORAGE_PATH] = " << storage_path << std::endl; std::cout << " ENV[PROGRAM_NTHREADS] = " << number_of_threads << std::endl; std::cout << " ENV[PROGRAM_TIMEOUT] = " << timeout << std::endl; return EXIT_SUCCESS; }
Building:
$ g++ envvar.cpp -o envvars.bin --std=c++1z -g -O0 -Wall -Wextra
Running 1:
$ ./envvars.bin [1] Environment variable PROGRAM_USER_KEY not set. ==== Program Runtime Configuration ======= ENV[PROGRAM_STORAGE_PATH] = /home/dummy/.data ENV[PROGRAM_NTHREADS] = 10 ENV[PROGRAM_TIMEOUT] = 1
Running 2:
- Set PROGRAM_USER_KEY environment variable for the current terminal session.
$ export PROGRAM_USER_KEY=87axk689fghyk $ ./envvars.bin [1] Environment vartiable set => PROGRAM_USER_KEY = 87axk689fghyk ==== Program Runtime Configuration ======= ENV[PROGRAM_STORAGE_PATH] = /home/dummy/.data ENV[PROGRAM_NTHREADS] = 10 ENV[PROGRAM_TIMEOUT] = 1 $ export PROGRAM_USER_KEY=void_password $ ./envvars.bin [1] Environment vartiable set => PROGRAM_USER_KEY = void_password ==== Program Runtime Configuration ======= ENV[PROGRAM_STORAGE_PATH] = /home/dummy/.data ENV[PROGRAM_NTHREADS] = 10 ENV[PROGRAM_TIMEOUT] = 1
Running 3:
- Use the env
/usr/bin/env
for setting program environment variable.
$ env PROGRAM_NTHREADS=15 ./envvars.bin
[1] Environment vartiable set => PROGRAM_USER_KEY = void_password
==== Program Runtime Configuration =======
ENV[PROGRAM_STORAGE_PATH] = /home/dummy/.data
ENV[PROGRAM_NTHREADS] = 15
ENV[PROGRAM_TIMEOUT] = 1
$ ./envvars.bin
[1] Environment vartiable set => PROGRAM_USER_KEY = void_password
==== Program Runtime Configuration =======
ENV[PROGRAM_STORAGE_PATH] = /home/dummy/.data
ENV[PROGRAM_NTHREADS] = 10
ENV[PROGRAM_TIMEOUT] = 1
Running 4:
- Set multiple environment varaibles with "env" Unix program.
$ env PROGRAM_TIMEOUT=2.551 PROGRAM_STORAGE_PATH=/var/www ./envvars.bin [1] Environment vartiable set => PROGRAM_USER_KEY = void_password ==== Program Runtime Configuration ======= ENV[PROGRAM_STORAGE_PATH] = /var/www ENV[PROGRAM_NTHREADS] = 10 ENV[PROGRAM_TIMEOUT] = 2.551
1.2 C-legacy Printf-like IO functions - format specifiers and examples
1.2.1 Puts
Print a string without any formatting - header: <cstdio> for C++ and <stdio.h> for C.
- puts - print a line to stdout followed by new line character.
int puts(const char*);
- fputs - print line to file.
int fputs(FILE* fd, const char*); // Example: print line to stdout - standard output std::fputs(" => hello world C++17 line\n", stdout); // Exmaple: print line to stderr - standard error output std::fputs(" => hello world C++17 line\n", stderr);
1.2.2 Printf-like Functions
The following functions are available in the header <cstdio> for C++, and in the header <stdio.h> for C.
- Printf - print to stdout - standard output.
int printf(const char* format, ...);
- fprintf - print to file
- stdout - file => standard output
- stderr - file => standard error output.
int fprintf(FILE* fd, const char* format, ...); // Example: Print to stdout, same as printf std::fprintf(stdout, " text = '%s' x = %.3f ", "square_root", 3.141598714); // Example: Print to stderr std::fprintf(stderr, " text = '%s' x = %.3f ", "square_root", 3.141598714);
- dprintf - print to file descriptor
int dprintf(int fd, const char* format, ...);
- sprintf - print to buffer - Note: function prone to buffer overlflow.
int sprintf(char* buffer, const char* format, ...);
- snprintf - print to buffer with bound checking - size is the maximum number of characters that can be print to the buffer.
int snprintf(char* buffer, size_t size, const char* format, ...);
1.2.3 Format String Specifiers for printf-like functions
Escacape sequences:
Sequence | Description |
---|---|
New Line | |
\n |
LF - Line Feed (Unix new line ) |
\r |
CR - Carriage Return |
Tabs | |
\t |
Horizontal Tab |
\v |
Vertical Tab |
Escape Characters | |
%% | Print a percent sign (%) |
\\ | Print backlash, aka backward slash |
\" | Print a single double quote |
Special Characters | |
\a |
Alert bell |
\b |
Backspace |
\c |
|
\f |
Form feed |
\e |
Escape |
Most useful specifiers:
Specifier | Type | Note |
---|---|---|
%s | char | character |
%d | int | decimal format |
%s | const char* | null-terminated array of characters (C "strings" for short). |
%X | int | hexadecimal format |
%ld | long | - |
%lld | long long | - |
%p | any pointer type | print the numeric value of any pointer in hexadecimal format. |
%zu | size_t | |
%td | ptrdiff_t | |
Comprehensive list of Specifiers:
Specifier | Description - Format Specifier for type: | Example |
---|---|---|
Misc | ||
%c | char - single character | |
%s | char* or const char* | |
%p | void* - Print address stored in a pointer in hexadecimal format. | |
Integers | ||
%u | unsigned integer | |
%d or %i | int - signed integer in decimal base | |
%o | int - signed integer in octal base | |
%x | int - signed integer in hexadecimal base (lower case) | |
%X | int - signed integer in hexadecimal base (upper case) | |
Long Types | ||
%ld | 'long' or 'long int' in decimal format | |
%lx | 'long' or 'long int' in hex format (lower case) | |
%lX | 'long' or 'long int' in hex format (upper case) | |
%lu | 'unsigned long' or 'size_t' in decimal format | |
%zx | 'unsigned long' or 'size_t' in hex format (lower case) | |
%zX | 'unsigned long' or 'size_t' in hex format (upper case) | |
%lld | 'long long' or 'long long int' in decimal format | |
%llu | 'unsigned long long' | |
Float Point | For float, double, and long double | |
%f | Print using standard notation | 12391.647808 |
%a | Print using hexadecimal notation | |
%e | Print using scientific notation (lower case 'e) | 1.239165e+04 |
%E | Print using scientific notation (upper case 'E') | 1.239165E+04 |
%g | Print using scientific notation only when needed (lower case 'e') | 1.239165e+04 |
%G | Print using scientific notation only when needed (upper case 'E') | 1.239165E+04 |
%.3f | Print using std. notation with 3 digits precision | 12391.649 |
%.3E | Print using scientific notation with 3 digits precision | 1.240E+04 |
WARNING: Despite the simplicity, printf is not type-safe and has some potential vulnerabilities:
1.2.4 Examples
Headers:
// For C use <stdio.h> #include <cstdio> // for printf #include <cmath> // sin, cos, exp
Print integer:
int x = 2951; >> printf(" => x = %d\n", x); => x = 2951 // Print integer in octal format >> printf(" => x = %o [Octal format - base 8]\n", x); => x = 5607 [Octal format - base 8] // Print integer in hexadecimal format (lower case) >> printf(" => x = 0x%x [Hexadecimal]\n", x); => x = 0xb87 [Hexadecimal] // Print integer in hexadecimal format (upper case) >> printf(" => x = 0x%X [Hexadecimal]\n", x); => x = 0xB87 [Hexadecimal]
Print address of a pointer
>> printf(" => address_of(x) = %p\n", &x) => address_of(x) = 0x7f0279845010
Print float point
Print float point in standard notation with any precision:
>> double PI = 4.0 * atan2(1, 1) (double) 3.1415927 // Print with any precision >> printf("PI = %f exp(3 * PI) = %f \n", PI, std::exp(3 * PI)); PI = 3.141593 exp(3 * PI) = 12391.647808
Print float point in standard notation with 3 digits precision:
// Print with 3 digits precision >> printf("PI = %.3f exp(3 * PI) = %.3f \n", PI, std::exp(3 * PI)); PI = 3.142 exp(3 * PI) = 12391.648
Print in scientific notation with any precision:
>> printf("PI = %e exp(3 * PI) = %e \n", PI, std::exp(3 * PI)); PI = 3.141593e+00 exp(3 * PI) = 1.239165e+04
Print in scientific notation with 3 digits precision:
>> printf("PI = %.3E exp(3 * PI) = %.3E \n", PI, std::exp(3 * PI)); PI = 3.142E+00 exp(3 * PI) = 1.239E+04
Print with scientific only when needed (for large numbers).
>> printf("PI = %G exp(2 + 3 * PI) = %G \n", PI, std::exp(2 + 3 * PI)); PI = 3.14159 exp(2 + 3 * PI) = 91562.6 >> printf("PI = %G exp(6 + 3 * PI) = %G \n", PI, std::exp(6 + 3 * PI)); PI = 3.14159 exp(6 + 3 * PI) = 4.99915E+06
Print with scientific only when needed with 4 digits precision:
>> printf("PI = %.4G exp(2 + 3 * PI) = %.4G \n", PI, std::exp(2 + 3 * PI)); PI = 3.142 exp(2 + 3 * PI) = 9.156E+04
Print with hexadecimal notation:
>> printf("PI = %a exp(PI) = %a \n", PI, std::exp(PI)); PI = 0x1.921fb54442d18p+1 exp(PI) = 0x1.724046eb09339p+4
Print table of float points:
>> for(double x = 0; x <= 10.0; x+= 1.0) printf("%8.3f %10.3f\n", x, exp(x)) 0.000 1.000 1.000 2.718 2.000 7.389 3.000 20.086 4.000 54.598 5.000 148.413 6.000 403.429 7.000 1096.633 8.000 2980.958 9.000 8103.084 10.000 22026.466 >> for(double x = 0; x <= 10.0; x+= 1.0) printf("%8.3f %10.2E\n", x, exp(x)) 0.000 1.00E+00 1.000 2.72E+00 2.000 7.39E+00 3.000 2.01E+01 4.000 5.46E+01 5.000 1.48E+02 6.000 4.03E+02 7.000 1.10E+03 8.000 2.98E+03 9.000 8.10E+03 10.000 2.20E+04 >> for(double x = 0; x <= 10.0; x+= 1.0) printf("%8.3f %10.5E\n", x, exp(x)) 0.000 1.00000E+00 1.000 2.71828E+00 2.000 7.38906E+00 3.000 2.00855E+01 4.000 5.45982E+01 5.000 1.48413E+02 6.000 4.03429E+02 7.000 1.09663E+03 8.000 2.98096E+03 9.000 8.10308E+03 10.000 2.20265E+04
Print table of string:
- file: main.cpp
#include <iostream> #include <string> #include <vector> int main() { using table_row = std::pair<std::string, double>; std::vector<table_row> table{ {"tristor", 10.23} ,{"voltage regulator 5 volts", 1.25} ,{"voltage regulator 12V", 9.50} ,{"IMU sensor", 108.23} ,{"Pressure transducer", 205.90} }; std::printf("\n ==>>> C++11 Table printing (First column right aligned) \n\n"); for(auto const& r: table) std::printf("%25s %10.2f\n", r.first.c_str(), r.second); std::printf("\n ==>>> C++17 Table printing (First column left aligned) \n\n"); for(auto const& [key, value]: table) std::printf("%-25s %10.2f\n", key.c_str(), value); }
Output:
==>>> C++11 Table printing (First column right aligned) tristor 10.23 voltage regulator 5 volts 1.25 voltage regulator 12V 9.50 IMU sensor 108.23 Pressure transducer 205.90 ==>>> C++17 Table printing (First column left aligned) tristor 10.23 voltage regulator 5 volts 1.25 voltage regulator 12V 9.50 IMU sensor 108.23 Pressure transducer 205.90
1.3 C-legacy File IO functions
Headers Files
- #include <stdio.h> for C
- #include <cstdio> for C++
Special File Streams
- stdout (FILE*) => Console/terminal standard output
- stderr (FILE*) => Console/terminal standard error output
- stdin (FILE*) => Console/terminal standard input
Fundamental Functions
- fopen() => Open a file returning a file stream (FILE*)
FILE *fopen(const char *pathname, const char *mode);
- fmemopen() => Associate a stream with a buffer. This function
allows applying file stream operations to a buffer.
- Documentation:
FILE* fmemopen( void* restrict buf /* Pointer to beginning of buffer */ , size_t size /* Buffer size */ , const char* restrict mode );
- fclose() => Closes a stream.
- "The fclose() function dissociates the named stream from its underlying file or set of functions. If the stream was being used for output, any buffered data is written first, using fflush(3)." (FreeBSD Manpage)
int fclose(FILE *stream)
- fflush() => Flushes a stream forcing to all data to be written to
the output.
- "For output streams, fflush() forces a write of all user-space buffered data for the given output or update stream via the stream's underlying write function. For input streams associated with seekable files (e.g., disk files, but not pipes or terminals), fflush() discards any buffered data that has been fetched from the underlying file, but has not been consumed by the application." (Linux Manpage)
int fflush(FILE *stream);
- ftell() => Get current file offset as number of bytes from the beginning of the file.
long ftell(FILE *stream);
- fseek() => Set current file offset as number of bytes from the beginning of the file.
int fseek(FILE *stream, long offset, int whence);
Streams and File Descriptors
- Special File Descriptors (Unix) with type (int)
- STDIN_FILENO
- STDERR_FILENO
- STDOUT_FILENO
- fileno() => Get file descriptor associated to a stream. (Note: It returns a file-descriptor as an integer.)
int fileno(FILE* stream);
- fdopen() => Associate a stream with a file descriptor. [UNIX-ONLY]
FILE* fdopen(int fd, const char* mode);
Text I/O - Input/Output
- fputs() => Write c-string (null-terminated '\0' array of characters) to stream.
int fputs(const char* s, FILE* stream)
- fputc() => Write byte or character to stream.
int putc(int c, FILE* stream);
- fgets() => Read a buffer from a stream.
char* fgets(char* s, int size, FILE* stream);
- fscanf() => Read formatted IO
int fscanf(FILE *stream, const char *format, ...);
- getline() => Read file from a stream.
ssize_t getline(char** lineptr, size_t* n, FILE* stream);
- getdelim() => Similar, to getline, but can use other delimiters.
ssize_t getdelim(char** lineptr, size_t* n, int delim, FILE* stream);
Binary I/O - Input/Output related
- fread() => Read binary from stream
size_t fread(void* ptr, size_t size, size_t nmemb, FILE* stream);
- fwrite() => Write binary data to stream
size_t fread(void* ptr, size_t size, size_t nmemb, FILE* stream);
1.4 Standard Library IO Class Hierarchy
C++ Standard Library IO Features:
- Object oriented and polymorphic
- Stateful
- The IO classes have internal states such as precision that float point will be printed; base which numbers will be print such as octal (base 8), hexadecimal (base 16) and decimal (default).
- Without exceptions
- The C++ IO does not have exceptions, it indicates error through error flags that must be checked by the calling code.
IO Standard Library IO Classes:
- ios - base class for streams
- istream - input stream base class
- ifstream => Input file stream
- istringstream => Input string stream
- ostream - output stream base class
- ofstream => Output file stream
- ostringstream => Output string stream
- iostream - input/output stream base class
- fstream => Input/Output file stream.
- sstringstream => String stream that can be used for reading and writing.
IO Global Objects
Output stream global objects (instances of std::ostream):
- std::cout - Standard output stream (stdout)
- std::cerr - Standard output error stream (stderr)
- std::clog
- std::wcout - Standard output stream (stdout) - wide character version.
- std::wcerr
- std::wclog
Input stream global objects (instances of std::istream):
- std::cin - Standard input stream (stdin)
- std::wcin - Standard input stream (stdin), wide character version.
Header Files
- <iostream> => Basic IO
- std::ostream, std::istream, std::cout, std::cerr, std::clog, std::wcerr
- <fstream> => File IO
- std::ifstream, std::ofstream, std::fstream
- <sstream> => String / Memory IO
- std::stringstream, std::istringstream, std::ostringstream
- <iomanip> => IO manipulation functions
- std::setfill, std::setprecision, std::setw, std::right, std::left, std::dec, std::hex, std::quoted (C++14)
Full references at:
ios + +----------------+--------------+ | | \|/ \|/ +-------+ +--------+ |istream| |ostream | +--+----+ +---+----+ | | \|/ +-----------+ \|/ +------------------>+ iostream +<------+--v--------+-----------+ | | +-----+-----+ | | | v v | v v v *std::cin ifstream | *std::cerr ofstream ostringstream istringstream | *std::cout | *std::clog ------v------ | | v v fstream stringstream
Statefullness
The C++ IO streams are stateful, they have internal states for IO configuration such as IO flags and numerical precision. In the following code, the IO manipulator std::boolalpha enables the IO flag std::ios_base::boolalpha that makes the IO stream print booleans as string instead of '0' or '1'. The flag can be disabled with the IO manipulator std::noboolalpha.
#include <iostream> #include <iomanip> >> std::cout << "flag1 = " << true << " flag2 = " << false << "\n"; flag1 = 1 flag2 = 0 >> >> std::cout << "flag1 = " << true << " flag2 = " << false << "\n"; flag1 = 1 flag2 = 0 >> // IO Manipulator that enables boolalpha flag which enables booleans // be print as strings instead of being print as numbers. >> std::cout << std::boolalpha; >> std::cout << "flag1 = " << true << " flag2 = " << false << "\n"; flag1 = true flag2 = false // Output continues in this way until the flag boolalpha is disabled. >> std::cout << "flag1 = " << true << " flag2 = " << false << "\n"; flag1 = true flag2 = false >> // Disable flag >> std::cout << std::noboolalpha; >> >> std::cout << "flag1 = " << true << " flag2 = " << false << "\n"; flag1 = 1 flag2 = 0
1.5 Standard Library IO Manipulators
- All mapipulators: documentation
Switch numeric representation of booleans
- std::boolapha
- => Print booleans as 'true' or 'false' rather than 0 or 1.
- std::noboolalpha
- => Print boolean as number true as 1 and false as 0.
Case manipulators:
- std::uppercase => Print everything as uppercase.
- std::nouppercase => Print without forcing uppercase.
Change numerical base used for printing integers (flag std::ios::basefield)
- std::dec => Print integers as decimal
- std::hex => Print integers as heaxdecimal
- std::oct => Print integers as octal
Float point number formatting: (flag: std::ios::floatfield)
- std::defaultflot (C++11)
- std::fixed
- Print float with fixed precision (fixed numbers of decimal places)
- std::scientific
- Print float with scientific notation
- std::hexfloat (C++11)
Float point precision:
- std::setprecision(int N)
Field adjusting (flag: std::ios::adjustfield)
- std::ios::left
- std::ios::right
- std::ios::internal
Flushing manipulators:
- std::flush
- Flush output stream, in other words, it forces the stream buffer to be written to the IO, for instance, it can force the buffer of a file output stream to be written to a file. TL;DR synchronize stream buffer with output.
- std::endl
- Insert new line character in the output stream
\n
and flushes the stream (std::flush).
- Insert new line character in the output stream
- std::unitbuf
- => Automatically flushes the output stream after each operation. Note: most output streams flushes by default.
- Sets the flag: std::ios_base::unitbuf.
- std::nounitbuf
- => Disables automatic output stream flushing.
- => Disables the flag: std::ios_base::unitbuf
Examples Float point IO manipulators
#include <iostream> #include <iomanip> #include <cmath> >> std::cout << "sqrt(pi) = " << std::sqrt(M_PI) << " x = " << 2.50 << "\n"; sqrt(pi) = 1.77245 x = 2.5 >> // Precision 3 decimal places >> std::cout << std::fixed << std::setprecision(3); >> std::cout << "sqrt(pi) = " << std::sqrt(M_PI) << " x = " << 2.50 << "\n"; sqrt(pi) = 1.772 x = 2.500 // Precision 6 decimal places >> std::cout << std::fixed << std::setprecision(6); >> std::cout << "sqrt(pi) = " << std::sqrt(M_PI) << " x = " << 2.50 << "\n"; sqrt(pi) = 1.772454 x = 2.500000 // Reset float IO >> std::cout << std::defaultfloat; >> std::cout << "sqrt(pi) = " << std::sqrt(M_PI) << " x = " << 2.50 << "\n"; sqrt(pi) = 1.77245 x = 2.5 >>
1.6 Standard Library IO stream redirection
1.6.1 Stdout (cout) and Stderr (cerr) stream redirection
Every process, has the following standard streams:
- stdin - Standard input (file descriptor 0) => encapsulated by global object std::cin (derived class of std::istream)
- stdout - Standard output (file descriptor 1) => encapsulated by
global object std::cout (derived class of std::ostream)
- Normal program console output should use std::cout.
- stderr - Standard error output (file descriptor 2) => encapsulated
by global object std::cerr (derived class of std::ostream)
- Diagnostics, error and logging should use std::cerr as this approach allows separating logging and diagnostics from program output and redirecting or discarding them.
- Note: Stream redirection is also possible on Windows. Althoug, it is not so usual as in Unix-like operating system due to low usage of terminal application in Windows.
- Note: A Windows application compiled for GUI subsystem will not display any output stream even if it is run from the terminal.
Example:
- File: stream-redirect.cpp
- Sample program that solves the equation f(x) = x^3 - a = 0 using newton raphson method.
#include <iostream> // std::cout, std::cerr ... #include <string> // std::stod #include <cmath> // std::sin, std::fabs, std::abs, ... #include <iomanip> // std::seprecision, std::fixed int main(int argc, char** argv) { // 1e-6 or 6 digits tolerance double tolerance = 0.000001; // Maximum number of iterationrs size_t itermax = 200; if(argc < 2) { std::cout << "Solves equation x^3 - a = 0 using Newton Raphson Method" << std::endl; std::cout << " Error: input parameter not provided" << std::endl; return EXIT_FAILURE; } //------- Parse Input Argument -------------- double a; try { a = std::stod(argv[1]); } catch (std::exception const& ex) { std::cerr << " [ERROR] " << ex.what() << std::endl; return EXIT_FAILURE; } double dx, x = a; std::cerr << std::fixed << std::setprecision(4); int w = 10; std::cerr << " [INFO] Solving equation f(x) = x^3 - a ; for a = " << a << "\n\n"; std::cerr << std::setw(w) << "Iter" << std::setw(w) << "x" << std::setw(w) << "AbsErr" << std::setw(w) << "RelErr%" << std::endl; // Find the cubic root by solving the following equation with // Newton Raphson method: // =>> F(x) = x^3 - a = 0 for(auto iter = 0; iter < itermax; iter++) { dx = (x * x * x - a ) / ( 2.0 * x * x); x = x - dx; std::cerr << std::setw(w) << iter << std::setw(w) << x << std::setw(w) << std::fabs(dx) << std::setw(w) << 100.0 * std::fabs(dx / x) << std::endl; if(std::fabs(dx) < tolerance * std::fabs(x)) { std::cout << " Cubic root of a = " << a << " ; is approximately " << x << " number of iteratios is " << iter + 1 << std::endl; return EXIT_SUCCESS; } } std::cerr << " [FAILURE] Root does not converge" << std::endl; return EXIT_FAILURE; }
Compiling:
# GCC $ g++ stream-redirect.cpp -o stream-redirect.bin -std=c++1z -g -O0 -Wall # Clang / LLVM $ clang++ stream-redirect.cpp -o stream-redirect.bin -std=c++1z -g -O0 -Wall
Running:
- Print all results - the standard streams stdcout (std::cout) and stderr are redirected to console by default.
- Note: Stream redirectiion peformed on Bash shell on Linux.
$ ./stream-redirection.bin 125.0 [INFO] Solving equation f(x) = x^3 - a ; for a = 125.0000 Iter x AbsErr RelErr% 0 62.5040 62.4960 99.9872 1 31.2680 31.2360 99.8977 2 15.6979 15.5701 99.1855 3 8.1026 7.5953 93.7396 4 5.0033 3.0993 61.9454 5 4.9984 0.0049 0.0985 6 5.0008 0.0025 0.0492 7 4.9996 0.0012 0.0246 8 5.0002 0.0006 0.0123 9 4.9999 0.0003 0.0062 10 5.0001 0.0002 0.0031 11 5.0000 0.0001 0.0015 12 5.0000 0.0000 0.0008 13 5.0000 0.0000 0.0004 14 5.0000 0.0000 0.0002 15 5.0000 0.0000 0.0001 Cubic root of a = 125 ; is approximately 5 number of iteratios is 16
- Discard stderr standard error stream by redirecting it to
/dev/null. Note: valid for Unix-like OSes (Linux, BSD, OSX, QNX …):
- $ ./program 2> /dev/null
$ ./stream-redirect.bin 125.0 2> /dev/null Cubic root of a = 125 ; is approximately 5 number of iteratios is 16
- Discard stdout - standard output by redirecting it to /dev/null
(Unix-like OSes)
- $ ./program > /dev/null
$ ./stream-redirect.bin 125.0 > /dev/null [INFO] Solving equation f(x) = x^3 - a ; for a = 125.0000 Iter x AbsErr RelErr% 0 62.5040 62.4960 99.9872 1 31.2680 31.2360 99.8977 2 15.6979 15.5701 99.1855 3 8.1026 7.5953 93.7396 4 5.0033 3.0993 61.9454 5 4.9984 0.0049 0.0985 6 5.0008 0.0025 0.0492 7 4.9996 0.0012 0.0246 8 5.0002 0.0006 0.0123 9 4.9999 0.0003 0.0062 10 5.0001 0.0002 0.0031 11 5.0000 0.0001 0.0015 12 5.0000 0.0000 0.0008 13 5.0000 0.0000 0.0004 14 5.0000 0.0000 0.0002 15 5.0000 0.0000 0.0001
- Discard both stdout and stderr by redirecting them to /dev/null
- ./prorgram > /dev/null 2>&1
$ ./stream-redirect.bin 125.0 > /dev/null 2>&1
- Redirect stdout - standard output stream to a file.
- ./program > stdout.log
- Only the stderr is printed, the stdout is redirected to the text file stdout.log
$ ./stream-redirect.bin 125.0 > stdout.log [INFO] Solving equation f(x) = x^3 - a ; for a = 125.0000 Iter x AbsErr RelErr% 0 62.5040 62.4960 99.9872 1 31.2680 31.2360 99.8977 2 15.6979 15.5701 99.1855 3 8.1026 7.5953 93.7396 4 5.0033 3.0993 61.9454 5 4.9984 0.0049 0.0985 6 5.0008 0.0025 0.0492 7 4.9996 0.0012 0.0246 8 5.0002 0.0006 0.0123 9 4.9999 0.0003 0.0062 10 5.0001 0.0002 0.0031 11 5.0000 0.0001 0.0015 12 5.0000 0.0000 0.0008 13 5.0000 0.0000 0.0004 14 5.0000 0.0000 0.0002 15 5.0000 0.0000 0.0001 # Check file $ cat stdout.log Cubic root of a = 125 ; is approximately 5 number of iteratios is 16
- Redirect stderr (std::cerr) - standard error stream to file.
- ./program 2> stderr.log
- Only the stdout is printed and the stderr is redirect to the text file stderr.log
$ ./stream-redirect.bin 125.0 2> stderr.log Cubic root of a = 125 ; is approximately 5 number of iteratios is 16 # Show first 5 lines of log file $ head -n 5 stderr.log [INFO] Solving equation f(x) = x^3 - a ; for a = 125.0000 Iter x AbsErr RelErr% 0 62.5040 62.4960 99.9872 1 31.2680 31.2360 99.8977 # Show last 5 lines of log file $ tail -n 5 stderr.log 11 5.0000 0.0001 0.0015 12 5.0000 0.0000 0.0008 13 5.0000 0.0000 0.0004 14 5.0000 0.0000 0.0002 15 5.0000 0.0000 0.0001
- Redirect both streams to different files: stdcout (std::cout) and
stderr (std:cerr)
- $ ./program > stdout.txt 2> stderr.txt
- The stdout (std::cout) is redirected to the file stdout.txt and the stderr (std::cerr) is redirected to the file stderr.txt
$ ./stream-redirect.bin 125.0 > stdout.txt 2> stderr.txt # Check stdout (std::cout) log file $ cat stdout.txt Cubic root of a = 125 ; is approximately 5 number of iteratios is 16 # Check stderr (std::cerr) log file $ tail -n 5 stderr.txt 11 5.0000 0.0001 0.0015 12 5.0000 0.0000 0.0008 13 5.0000 0.0000 0.0004 14 5.0000 0.0000 0.0002 15 5.0000 0.0000 0.0001
- Redirect both streams to the same log file.
- ./program > log.txt 2>&1
$ ./stream-redirect.bin 125.0 > log.txt 2>&1 $ cat log.txt [INFO] Solving equation f(x) = x^3 - a ; for a = 125.0000 Iter x AbsErr RelErr% 0 62.5040 62.4960 99.9872 1 31.2680 31.2360 99.8977 2 15.6979 15.5701 99.1855 3 8.1026 7.5953 93.7396 4 5.0033 3.0993 61.9454 5 4.9984 0.0049 0.0985 6 5.0008 0.0025 0.0492 7 4.9996 0.0012 0.0246 8 5.0002 0.0006 0.0123 9 4.9999 0.0003 0.0062 10 5.0001 0.0002 0.0031 11 5.0000 0.0001 0.0015 12 5.0000 0.0000 0.0008 13 5.0000 0.0000 0.0004 14 5.0000 0.0000 0.0002 15 5.0000 0.0000 0.0001 Cubic root of a = 125 ; is approximately 5 number of iteratios is 16
Capturing process stdout and stderr with Python subprocess module
Run C++ program stream-redirect.bin as subprocess:
$ python3 Python 3.6.8 (default, Mar 21 2019, 10:08:12) [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> >>> import subprocess as sp # Process result >>> r = sp.run(["./stream-redirection.bin", "125"], stdout = sp.PIPE, stderr = sp.PIPE) >>>
Check status code:
- Process status code => Numeric value returned by int main(….){ … }
- 0 => Status code means successful return
>>> r.returncode 0
Check captured stdout:
# =============== Stdout => Standarde Output Stream =====================# >>> type(r.stdout) <class 'bytes'> >>> r.stdout b' Cubic root of a = 125 ; is approximately 5 number of iteratios is 16\n' >>> >>> print(r.stdout.decode('utf-8')) Cubic root of a = 125 ; is approximately 5 number of iteratios is 16
Check captured stderr:
# => Type is a byte array >>> type(r.stderr) <class 'bytes'> # => Type is a python string >>> type(r.stderr.decode('utf-8')) <class 'str'> >>> >>> print(r.stderr.decode('utf-8')) [INFO] Solving equation f(x) = x^3 - a ; for a = 125.0000 Iter x AbsErr RelErr% 0 62.5040 62.4960 99.9872 1 31.2680 31.2360 99.8977 2 15.6979 15.5701 99.1855 ... ... .... .... ... ... ... .... .... 15 5.0000 0.0000 0.0001
1.6.2 Stdin (cin) stream redirection
File: input-redirection.cpp
#include <iostream> // Import: std::cout, std::cin, std::cerr #include <iomanip> // Import: std::setw #include <string> // Import: std::string #include <functional> // Import: std::function int main(int argc, char** argv) { if(argc > 1 && std::string(argv[1]) == "--help") { std::cout << "Count line numbers of some file." << std::endl; std::cout << " Usage: \n" << " $ " << argv[0] << " < file.txt \n" << " $ cat file.txt | " << argv[0] << "\n\n"; // Early return return 0; } std::string line; long counter = -1; // Self-executable lambda used for returning a value from the if-else bool showLine = [&]{ if(argc > 1 && std::string(argv[1]) == "--show-line") return true; return false; }(); std::cerr << std::boolalpha << " [INFO] showLine = " << showLine << std::endl; /* Note: The comma opeator evaluates first (counter++) and then * evaluates std::getline(std::cin, line) returning its value. */ while(counter++, std::getline(std::cin, line)){ if(showLine) { std::cout << "\n" << std::right << std::setw(4) << counter << " " << std::left << std::setw(8) << line; } } std::cout << "\n\n File has " << counter << " lines " << std::endl; return 0; }
Compiling:
# Clang $ clang++ input-redirection.cpp -o input-redirection.bin -std=c++1z -g -O0 -Wall # GCC $ gcc++ input-redirection.cpp -o input-redirection.bin -std=c++1z -g -O0 -Wall
Usage:
Display help:
$ ./input-redirection.bin --help Count line numbers of some file. Usage: $ ./input-redirection.bin < file.txt $ cat file.txt | ./input-redirection.bin
Count number of line in input stream stdin (std::cin) typed by user:
- Note: When the user is done, he types CTRL + D to send EOF (End Of File) character to process.
$ ./input-redirection.bin [INFO] showLine = false DXY index VIX implied volatility VAR VALUE AT RISK standard deviation commodity prices monte carlo simulations stochastic volatility Finaly user types CTRL + D when he is done to send EOF signal character to process. File has 7 lines
Redirect output stream (stdout) of echo program to input stream (stdin) of input-redirection.bin:
$ echo -e " line0\n line1\n line2" line0 line1 line2 $ echo -e " line0\n line1\n line2" | ./input-redirection.bin [INFO] showLine = false File has 3 lines $ echo -e " line0\n line1\n line2" | ./input-redirection.bin --show-line [INFO] showLine = true 0 line0 1 line1 2 line2 File has 3 lines
Redirect file to input stream (stdin) of process input-redirection.bin:
# Show test file #------------------------------------------------- $ cat /etc/filesystems ext4 ext3 ext2 nodev proc nodev devpts iso9660 vfat hfs hfsplus * # Redirect test file to process' input stream without displaying lines #------------------------------------------------- $ ./input-redirection.bin < /etc/filesystems [INFO] showLine = false File has 10 lines # Redirect test file to process' input stream displaying its lines #------------------------------------------------- $ ./input-redirection.bin --show-line < /etc/filesystems [INFO] showLine = true 0 ext4 1 ext3 2 ext2 3 nodev proc 4 nodev devpts 5 iso9660 6 vfat 7 hfs 8 hfsplus 9 * File has 10 lines
Input redirection with Python Subprocess Module
$ python3 Python 3.6.8 (default, Mar 21 2019, 10:08:12) [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess as sp >>> >>> input = " ASM\n RUST \nC++ \n Lisp \n Ocaml\n Haskell\n --- Scala" >>>
Redirect Python input string to process:
>>> r = sp.run(["./input-redirection.bin"], stdout = sp.PIPE, stderr = sp.PIPE, input = input.encode()) >>> r CompletedProcess(args=['./input-redirection.bin'], returncode=0, stdout=b'\n\n File has 7 lines \n', stderr=b' [INFO] showLine = false\n')
Proces status code: numeric value returned by int main(…) function.
>>> r.returncode 0
Process stdout:
>>> r.stdout b'\n\n File has 7 lines \n' >>> >>> print(r.stdout.decode()) File has 7 lines
Proces stderr:
>>> r.stderr.decode()
' [INFO] showLine = false\n'
>>>
1.7 Reading data from generic Input Stream - read data from file, console and memory
File: readInventory.cpp
#include <iostream> // std::cout, std::cin, std::istream, std::ostream ... #include <fstream> // std::ifstream, std::ofstream #include <string> #include <sstream> // Import: std::stringstream #include <iomanip> // Import: std::quoted #include <vector> struct InventoryItem{ /// Database id unsigned int id; /// Produce name std::string name; /// Price per unit double price; /// Quanity unsigned int quantity; InventoryItem(): id{0}, name{""}, price{0.0}, quantity{0} { } }; using Inventory = std::vector<InventoryItem>; void showInventory(Inventory const& inventory); Inventory readInventory(std::istream& is); Inventory readInventoryFromFile(std::string const& file); extern const char* sampleData; int main(int argc, char** argv) { if(argc < 2) { std::cout << "Usage: " << "\n" << "$ " << argv[0] << " file [FILE] " << "\n" << "$ " << argv[0] << " memory" << "\n" << "$ " << argv[0] << " console" << "\n"; return EXIT_FAILURE; } std::string cmd = argv[1]; // Self test - read memory stream if(cmd == "memory") { auto ss = std::stringstream{sampleData}; auto inventory = readInventory(ss); showInventory(inventory); return EXIT_SUCCESS; } // Read user file if(argc == 3 && cmd == "file") { std::string file = argv[2]; std::cout << " Reading file: " << file << std::endl; Inventory inventory = readInventoryFromFile(file); showInventory(inventory); return EXIT_SUCCESS; } // Read console if(cmd == "console") { std::cout << "Type the inventory and then type Ctlr + D when you are done." << std::endl; auto inventory = readInventory(std::cin); std::cout << " ---- Inventory Value ---------" << std::endl; showInventory(inventory); return EXIT_SUCCESS; } std::cerr << " [ERROR] Invalid command line option." << std::endl; return EXIT_FAILURE; } // --------------------------------------------------------------// const char* sampleData = R"( 297 "Current sensor" 9.60 300 871 "Pressure transducer" 90.00 50 751 "Temperature SPI sensor" 40.25 20 652 "Gyroscope" 220.50 45 )"; void showInventory(Inventory const& inventory) { std::cout << std::setprecision(2) << std::fixed; std::cout << std::right << std::setw(5) << "ID" << std::right << std::setw(5) << " " << std::left << std::setw(30) << "Name" << std::right << std::setw(10) << "Price" << std::right << std::setw(12) << "Quantity" << std::right << std::setw(12) << "Value" << "\n"; double total = 0.0; for(auto const& item: inventory) { std::cout << std::right << std::setw(5) << item.id << std::right << std::setw(5) << " " << std::left << std::setw(30) << item.name << std::right << std::setw(10) << item.price << std::right << std::setw(12) << item.quantity << std::right << std::setw(12) << item.quantity * item.price << "\n"; total += item.quantity * item.price; } std::cout << "\n Total inventory value: " << total << "\n"; } /** Read inventory from any input stream (derived class from std::istream) such * as std::ifstream, std::cin, std::sstringstream. */ Inventory readInventory(std::istream& is) { InventoryItem currentItem; std::vector<InventoryItem> inventory; std::string line; std::stringstream sline; while(std::getline(is, line)) { if(line == "") { continue; } // Reset sline status flags sline.clear(); // Reset the value of sline stringstream sline.str(line); // Read line fields sline >> currentItem.id >> std::quoted(currentItem.name) >> currentItem.price >> currentItem.quantity; // Check error flags and stop on error if(!sline) { auto error_message = std::string("Error: invalid line: ") + line; throw std::runtime_error(error_message); } inventory.push_back(currentItem); } return inventory; } Inventory readInventoryFromFile(std::string const& file) { auto ifs = std::ifstream(file); if(!ifs) { std::cerr << " Error: file not found." << std::endl; std::terminate(); } return readInventory(ifs); }
File: inventory1.txt
200 "diode" 3.40 100 300 "blue led" 5.10 200 100 "power supply 5 Volts" 20.50 15 256 "battery 12 volts pack" 12.50 50 901 "light sensor" 60.00 40
Building:
$ g++ readInventory.cpp -o readInventory.bin -std=c++1z -O0 -g
Show options:
$ ./readInventory.bin
Usage:
$ ./readInventory.bin file [FILE]
$ ./readInventory.bin memory
$ ./readInventory.bin console
Read and display inventory data from memory (std::stringstream):
$ ./readInventory.bin memory ID Name Price Quantity Value 297 Current sensor 9.60 300 2880.00 871 Pressure transducer 90.00 50 4500.00 751 Temperature SPI sensor 40.25 20 805.00 652 Gyroscope 220.50 45 9922.50 Total inventory value: 18107.50
Read and display inventory data from console (stdin stream), the data is typed by the user and when he is done, he types Ctrl + D sending EOF (End Of File) character to the process:
$ ./readInventory.bin console Type the inventory and then type Ctlr + D when you are done. 982 "IMU 6 axis" 300.0 25 751 "Temperature sensor I2C" 45.0 200 231 "Current transducer XYZWK" 78.50 35 ---- Inventory Value --------- ID Name Price Quantity Value 982 IMU 6 axis 300.00 25 7500.00 751 Temperature sensor I2C 45.00 200 9000.00 231 Current transducer XYZWK 78.50 35 2747.50 Total inventory value: 19247.50
Read and show inventory data obtained through stream redirection:
$ cat inventory1.txt | ./readInventory.bin console Type the inventory and then type Ctlr + D when you are done. ---- Inventory Value --------- ID Name Price Quantity Value 200 diode 3.40 100 340.00 300 blue led 5.10 200 1020.00 100 power supply 5 Volts 20.50 15 307.50 256 battery 12 volts pack 12.50 50 625.00 901 light sensor 60.00 40 2400.00 Total inventory value: 4692.50
Read and display inventory data from file:
$ ./readInventory.bin file inventory1.txt Reading file: inventory1.txt ID Name Price Quantity Value 200 diode 3.40 100 340.00 300 blue led 5.10 200 1020.00 100 power supply 5 Volts 20.50 15 307.50 256 battery 12 volts pack 12.50 50 625.00 901 light sensor 60.00 40 2400.00 Total inventory value: 4692.50
1.8 Reading ELF binary files
1.8.1 Overview
Most Unix-like operating systems such as Linux, Android, BSD Variants, AIX and IRIX use ELF Executable Linkable Format binary file format for their executables and shared libraries (aka shared object). While the following code is specific for ELF files, it can be reused with any other type of binary file if the layout is known.
- Note: Although Mac OSX is an Unix-like operating system, it does not use ELF file format for its executables, instead it uses Mach-O file format.
Parts of an ELF object:
- File Header
- Section Table*
- Program Header Table
- Sections and Segments
The most relevant sections for an ELF object-code file are:
Section | Description |
---|---|
.text | Contains the compiled machine code. |
.data | Initialized data |
.bss | Non-initialized data |
.got | Global Offset Table |
.debug | Debug Symbols |
1.8.2 Checking an ELF test file
Identify an ELF file:
$ file /bin/bash /bin/bash: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 3.2.0, BuildID[sha1]=01497fbe04f926fce1493834b3d208f82ab29788, stripped, too many notes (256)
Check magic number
The first bytes must be \0x7f ELF
or the sequence of bytes: 0x7f 0x45
0x4c 0x46.
$ od -t x1 -N 8 /bin/bash 0000000 7f 45 4c 46 02 01 01 00 0000010 $ hexdump -C /bin/bash | head -n 2 00000000 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............| 00000010 03 00 3e 00 01 00 00 00 b0 e4 02 00 00 00 00 00 |..>.............|
Show ELF header:
$ readelf -h /bin/bash ELF Header: Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: DYN (Shared object file) Machine: Advanced Micro Devices X86-64 Version: 0x1 Entry point address: 0x2e4b0 Start of program headers: 64 (bytes into file) Start of section headers: 1192776 (bytes into file) Flags: 0x0 Size of this header: 64 (bytes) Size of program headers: 56 (bytes) Number of program headers: 11 Size of section headers: 64 (bytes) Number of section headers: 30 Section header string table index: 29
Show sections:
$ readelf --sections /bin/bash There are 30 section headers, starting at offset 0x123348: Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align [ 0] NULL 0000000000000000 00000000 0000000000000000 0000000000000000 0 0 0 [ 1] .interp PROGBITS 00000000000002a8 000002a8 000000000000001c 0000000000000000 A 0 0 1 [ 2] .note.ABI-tag NOTE 00000000000002c4 000002c4 0000000000000020 0000000000000000 A 0 0 4 [ 3] .note.gnu.propert NOTE 00000000000002e8 000002e8 0000000000000030 0000000000000000 A 0 0 8 [ 4] .note.gnu.build-i NOTE 0000000000000318 00000318 0000000000000024 0000000000000000 A 0 0 4 [ 5] .gnu.hash GNU_HASH 0000000000000340 00000340 00000000000049ac 0000000000000000 A 6 0 8 [ 6] .dynsym DYNSYM 0000000000004cf0 00004cf0 000000000000de18 0000000000000018 A 7 1 8 [ 7] .dynstr STRTAB 0000000000012b08 00012b08 00000000000092be 0000000000000000 A 0 0 1 [ 8] .gnu.version VERSYM 000000000001bdc6 0001bdc6 0000000000001282 0000000000000002 A 6 0 2 [ 9] .gnu.version_r VERNEED 000000000001d048 0001d048 00000000000000b0 0000000000000000 A 7 2 8 [10] .rela.dyn RELA 000000000001d0f8 0001d0f8 000000000000da10 0000000000000018 A 6 0 8 [11] .rela.plt RELA 000000000002ab08 0002ab08 0000000000001470 0000000000000018 AI 6 23 8 [12] .init PROGBITS 000000000002bf78 0002bf78 0000000000000017 0000000000000000 AX 0 0 4 [13] .plt PROGBITS 000000000002bf90 0002bf90 0000000000000db0 0000000000000010 AX 0 0 16 [14] .text PROGBITS 000000000002cd40 0002cd40 00000000000a59e1 0000000000000000 AX 0 0 16 [15] .fini PROGBITS 00000000000d2724 000d2724 0000000000000009 0000000000000000 AX 0 0 4 [16] .rodata PROGBITS 00000000000d2740 000d2740 0000000000018fd8 0000000000000000 A 0 0 32 [17] .eh_frame_hdr PROGBITS 00000000000eb718 000eb718 00000000000042dc 0000000000000000 A 0 0 4 ... ... ... ... ... ... ... ... ... ... ... ... ... ... Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific)
1.8.3 Sample program
File: read_elf.cpp
#include <iostream> #include <fstream> #include <cstdint> #include <vector> #include <iomanip> #include <cassert> #include <map> using ByteArray = std::vector<std::uint8_t>; /** Print byte array as string and non-printable chars as hexadecimal */ std::ostream& operator<<(std::ostream& os, const ByteArray& str); bool operator==(ByteArray const& rhs, ByteArray const& lhs); template<typename T> auto readbin_at(std::istream& is, long offset) -> T; template<typename T> void show_num_at(std::istream& is, long offset, const char* name); extern std::map<uint8_t, const char*> isa_type_database; enum class Endianess { little_endian = 1 , big_endian = 2 }; enum WordSize { word32bits = 1 ,word64bits = 2 }; int main() { const char* elf_file = "/usr/bin/bash"; // Create input stream object to read ELF binary file auto ifs = std::ifstream (elf_file, std::ios::in | std::ios::binary ); if(!ifs){ std::cerr << " [ERROR] Unable to open file." << std::endl; return EXIT_FAILURE; } // ==== Read magic header 0x7F ELF at offset 0x00 ===============================// ByteArray elf_header(4, 0x00); // Read 4 bytes at offset 0x00 ifs.read(reinterpret_cast<char*>(&elf_header[0]) , elf_header.size() * sizeof (std::uint8_t)); std::cout << " elf_header = " << elf_header << std::endl; // Check wheter magic bytes are equal to '0x7f E L F' if(elf_header != ByteArray{ 0x7f, 0x45, 0x4c, 0x46 }) { std::cerr << " Error: not an ELF unix executable file." << std::endl; std::exit(EXIT_FAILURE); } // ==== Read field EI_CLASS at 0x04 ============================================// // auto ei_class = readbin_at<uint8_t>(ifs, 0x04); printf(" ei_class = 0x%X \n", ei_class); WordSize wordsize = static_cast<WordSize>(ei_class); printf(" Executable type (word size): %s \n", (ei_class == 2 ? "64 bits" : " 32 bits")); //==== Read field EI_DATA at offset 0x05 ====================================// auto ei_data = readbin_at<uint8_t>(ifs, 0x05); printf(" ei_data = 0x%X \n", ei_data); Endianess endianess = static_cast<Endianess>(ei_data); if(endianess == Endianess::big_endian) printf(" => Processor is BIG ENDIAN \n"); if(endianess == Endianess::little_endian) printf(" => Processor is LITTLE ENDIAN\n"); //==== Read field EI_OSABI at offset 0x07 ======================================// // auto ei_osabi = readbin_at<uint8_t>(ifs, 0x07); printf(" ei_osabi = 0x%X \n", ei_osabi); //=== Read ISA - Instruction Set Architecture type ===========================// auto ei_machine = readbin_at<uint8_t>(ifs, 0x12); printf(" ei_machine = 0x%X \n", ei_machine); printf(" Architecture(ISA): %s \n", isa_type_database.at(ei_machine)); //=== Read e_phnum, e_shentsize, e_shnum and e_shstrndx ====================// // Note: assumes ELF 64 bits // auto e_shnum = readbin_at<uint16_t>(ifs, 0x3C); printf(" Number of sections of the ELF file = %d \n", e_shnum); show_num_at<uint32_t>(ifs, 0x30, "e_hsize"); show_num_at<uint16_t>(ifs, 0x36, "e_shentsize"); show_num_at<uint16_t>(ifs, 0x38, "e_phnum"); show_num_at<uint16_t>(ifs, 0x3A, "e_shentsize"); show_num_at<uint16_t>(ifs, 0x3C, "e_shnum"); //=== Read entry point field ================================================// uint64_t ei_entry; if(wordsize == WordSize::word32bits) ei_entry = readbin_at<uint32_t>(ifs, 0x18); if(wordsize == WordSize::word64bits) ei_entry = readbin_at<uint64_t>(ifs, 0x18); printf(" Entry point 0x%X \n", ei_entry); return 0; } // ***************************************************************// std::ostream& operator<<(std::ostream& os, const ByteArray& str) { for(const auto& ch: str) { if(std::isprint(ch)) os << ch << ""; else os << "\\0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(ch) << " " << std::dec; } return os; } bool operator==(ByteArray const& rhs, ByteArray const& lhs) { if(rhs.size() != lhs.size()) { return false; } return std::equal(rhs.begin(), rhs.end(), lhs.begin()); } /** @brief Read some value T at some offset of a binary input stream. * * @tparam T - type which will be read * @param is - Input stream opened in binary mode * @param offset - Offset or position where data will be read. */ template<typename T> auto readbin_at(std::istream& is, long offset) -> T { T value{}; if(offset > 0) { is.seekg(offset); } is.read(reinterpret_cast<char*>(&value), sizeof (T)); return value; } template<typename T> void show_num_at(std::istream& is, long offset, const char* name) { T data = readbin_at<T>(is, offset); printf(" => Field [%s] = %d \n", name, data); } // Instruction set ISA type database std::map<uint8_t, const char*> isa_type_database = { {0x00, "No specified"} ,{0x02, "Sparc"} ,{0x03, "x86"} ,{0x08, "MIPS"} ,{0x14, "Power PC"} ,{0x16, "S390"} ,{0x28, "ARM"} ,{0x2A, "SuperH"} ,{0x3E, "x86-64"} ,{0xB7, "AArch64"} ,{0xF3, "RISC-V"} };
Compile:
$ g++ read_elf.cpp -o read_elf.bin -std=c++1z -Wall -g -O0
Running and sample output:
$ ./read_elf.bin elf_header = \0x7f ELF ei_class = 0x2 Executable type (word size): 64 bits ei_data = 0x1 => Processor is LITTLE ENDIAN ei_osabi = 0x0 ei_machine = 0x3E Architecture(ISA): x86-64 Number of sections of the ELF file = 30 => Field [e_hsize] = 0 => Field [e_shentsize] = 56 => Field [e_phnum] = 11 => Field [e_shentsize] = 64 => Field [e_shnum] = 30 Entry point 0x2E4B0
Highlights:
Open file in binary mode:
auto ifs = std::ifstream (elf_file, std::ios::in | std::ios::binary );
Function for reading data at some offset from a input stream opened in binary mode.
/** @brief Read some value T at some offset of a binary input stream. * * @tparam T - type which will be read * @param is - Input stream opened in binary mode * @param offset - Offset or position where data will be read. */ template<typename T> auto readbin_at(std::istream& is, long offset) -> T { T value{}; if(offset > 0) { is.seekg(offset); } is.read(reinterpret_cast<char*>(&value), sizeof (T)); return value; }
Relevant Member function of class std::istream for binary file:
- seekg => Set current position/offset of input stream.
istream& istream::seekg (streampos offset);
- read => Read n bytes from input stream.
istream& istream::read (char* buffer, streamsize buffer_size);
1.8.4 References
- Exectuable Linkable Format (ELF)
- Executable Linkable Format - wikpedia
- Executable and Linkable Format 101 - Part 1 Sections and Segments
- The 101 of ELF files on Linux: Understanding and Analysis
- Chapter 8. Behind the process
- C compiler. Memory Map. Program in RAM
- Analyzing the Linker Map file with a little help from the ELF and the DWARF
- Lab 03 - Executables. Static Analysis
- Inside ELF Symbol Table
1.9 Reading whole input stream to string
Note: It is advisable to not use the function istreamToString with large files >= 1GB, otherwise it can cost 1GB of RAM. Large files should be processed chunk-by-chunk in order to not consume an unnecessary amount of memory.
#include <iostream> #include <string> #include <sstream> #include <fstream> /** @brief Read whole input stream to a string and returns it * Requires headers: <iostream> and <sstream> */ std::string istreamToString(std::istream& is){ if(is.bad()){ throw std::runtime_error("Error: stream has errors."); } std::stringstream ss; ss << is.rdbuf(); return ss.str(); }
Read std::cin stream - console (stdin)
// Use types a text and types CTRL + D when he is done sending EOF character >> auto text = istreamToString(std::cin); A stdin input stream works like a "console file". In UNIX everything has a file interface, works with a file API, everything is a file. Type CTRL + D to send EOF and close this stream. >> std::cout << "text = " << text2 << std::endl; text = A stdin input stream works like a "console file". In UNIX everything has a file interface, works with a file API, everything is a file. Type CTRL + D to send EOF and close this stream.
Read file input stream:
>> auto ifs = std::ifstream("/proc/sys/kernel/osrelease") (std::basic_ifstream<char, std::char_traits<char> > &) @0x7f3edbc0a688 >> auto os_release = istreamToString(ifs) (std::basic_string<char, std::char_traits<char>, std::allocator<char> > &) "4.20.16-100.fc28.x86_64 " >> std::cout << " =>> Linux-OS Release = " << os_release << std::endl; =>> Linux-OS Release = 4.20.16-100.fc28.x86_64
Read stringstream - memory stream.
>> std::stringstream ms; >> ms << " UNIX - Linux OS2 Mainframes AIX VxWorks QNX "; >> ms << "\n network TCP IP UDP SOCKET packets"; >> ms << " \n binary bits octets speed 3G 4G 5G business flow currency"; >> ms << " \n technology price technology technology technology"; >> ms << std::flush; // Type of txt => std::string >> auto txt = istreamToString(ms); >> std::cout << "text = \n" << txt ; text = UNIX - Linux OS2 Mainframes AIX VxWorks QNX network TCP IP UDP SOCKET packets binary bits octets speed 3G 4G 5G business flow currency
1.10 Reading whole file to string
The following function read a given text file content to a string.
#include <iostream> #include <string> #include <sstream> #include <fstream> // Requires: <string>, <stream>, <sstream> std::string readFile(std::string const& file) { std::ifstream is(file); if( !is.good() ){ throw std::runtime_error("Error: stream has errors."); } std::stringstream ss; ss << is.rdbuf(); return ss.str(); }
Testing in ROOT repl:
root [11] std::string content = readFile("/etc/filesystems") (std::string &) "xfs ext4 ext3 ext2 nodev proc nodev devpts iso9660 vfat hfs hfsplus * " root [13] std::cout << " Content of file /etc/filesystems = \n\n" << content << Content of file /etc/filesystems = xfs ext4 ext3 ext2 nodev proc nodev devpts iso9660 vfat hfs hfsplus *
1.11 IO - Insertion << and extraction >> operators
1.11.1 Operators insertion insertion for std::vector<T>
- Code: Wandbox Online Compiler
Operator: (<<)
#include <iostream> #include <vector> #include <string> template<typename Element> std::ostream& operator<<(std::ostream& os, std::vector<Element> const& xs ) { os << "[" << xs.size() << "]( "; for(const auto& x: xs) os << x << " "; return os << " )"; }
Main function:
std::vector<double> xs; xs = {}; std::cout << "xs = " << xs << std::endl; xs = {1.0, 5.0, 4.6, 1e4, 9.8}; std::cout << "xs = " << xs << std::endl; std::vector<int> xa = {1, 3, 4, 5, 6}; std::cout << "xa = " << xa << "\n"; auto words = std::vector<std::string>{"hello", "world", "C++", "new", "machine"}; std::cout << "words = " << words << "\n"; return 0;
Output:
xs = [0]( ) xs = [5]( 1 5 4.6 10000 9.8 ) xa = [5]( 1 3 4 5 6 ) words = [5]( hello world C++ new machine )
1.11.2 Operators insertion and extraction for used defined-type
This example shows how to overload the operators insertion (<<) and extraction (>>) for an user defined type. Note: those operators are not class methods, they are free functions.
- File: file:src/insertionExtractionOverload.cpp
- Online Compile: https://wandbox.org/permlink/97fcclubA7KHWEry (No console input support)
Compiling and running:
$ clang++ inputOutput.cpp -o inputOutput.bin -std=c++1z -O0 -Wall && ./inputOutput.bin
Class Product (file: Product.hpp):
class Product{ private: int m_id; std::string m_name; double m_price; public: Product(); Product(int id, std::string const& name, double price); int Id() const; std::string Name() const; double Price() const; void showProduct() const; friend auto operator<<(std::ostream& os, Product const& prod) -> std::ostream&; friend auto operator>>(std::istream& is, Product& prod) -> std::istream&; };
Methods implementation (file: Product.hpp)
Constructors:
Product::Product(int id, std::string const& name, double price): m_id(id), m_name(name), m_price(price) { } Product::Product(): Product(-1, "unnamed", 0.0) { }
Operator insertion (<<) for printing a Product object to an output stream, it means any derived class of std::ostream or std::cout, std::stringstream, std::fstream.
- Note: This operator is not a method of the class Product, it is just a function of two arguments.
auto operator<<(std::ostream& os, Product const& prod) -> std::ostream& { os << " " << prod.m_id << " " << std::quoted(prod.m_name) << " " << prod.m_price; return os; }
Operator extraction (>>) for reading a product object from an input stream, it means, any derived class of the class std::istream such as std::cin, std::fstream, std::sstream and so on.
auto operator>>(std::istream& is, Product& prod) -> std::istream& { return is >> prod.m_id >> std::quoted(prod.m_name) >> prod.m_price; }
Main function:
Create sample products:
Product p1{200, "Arabica Coffee", 4.50}; Product p2{300, "Orange 1kg ", 10.0}; Product p3{126, "XYWZ soft drink", 15.60};
Experiment 1: print products to stdout.
std::puts(" >>> EXPERIMENT 1 == Print Products ==========="); std::cout << "Product p1 = {" << p1 << " }" << std::endl; std::cout << "Product p2 = {" << p2 << " }" << std::endl; std::cout << "Product p3 = {" << p3 << " }" << std::endl;
Output:
>>> EXPERIMENT 1 == Print Products =========== Product p1 = { 200 "Arabica Coffee" 4.5 } Product p2 = { 300 "Orange 1kg " 10 } Product p3 = { 126 "XYWZ soft drink" 15.6
Experiment 2: Print Product to memory stream (memory file)
std::puts(" >>> EXPERIMENT 2 == Print Products to memory stream (memory file) ===="); std::stringstream ss; ss << p1 << "\n" << p2 << "\n" << p3; std::cout << "ss = \n" << ss.str() << std::endl;
Output:
>>> EXPERIMENT 2 == Print Products to memory stream (memory file) ==== ss = 200 "Arabica Coffee" 4.5 300 "Orange 1kg " 10 126 "XYWZ soft drink" 15.6
Experiment 3: Read product from memory stream.
std::puts(" >>> EXPERIMENT 3 == Read products from memory stream ===="); Product pr; ss >> pr; std::cout << " pr1 = " << pr << std::endl; ss >> pr; std::cout << " pr2 = " << pr << std::endl; ss >> pr; std::cout << " pr3 = " << pr << std::endl;
Output:
>>> EXPERIMENT 3 == Read products from memory stream ==== pr1 = 200 "Arabica Coffee" 4.5 pr2 = 300 "Orange 1kg " 10 pr3 = 126 "XYWZ soft drink" 15.6
Experiment 4: Read products from console (user types the products) and types Ctrl + D sending EOF (End Of File) character when he is done.
std::puts(" >>> EXPERIMENT 4 == Read products from console ===="); Product prod; while(!std::cin.eof()){ std::cout << "Enter product: "; std::cin >> prod; std::cout << " prod = " << prod << " ; " << " price = " << prod.Price() << std::endl; }
Output:
Enter product: 451 Lemon 6.7 prod = 451 "Lemon" 6.7 ; price = 6.7 Enter product: 234 "Fresh atum 1kg" 25.90 prod = 234 "Fresh atum 1kg" 25.9 ; price = 25.9 Enter product: 461 "Cappucino coffee" 25.60 prod = 461 "Cappucino coffee" 25.6 ; price = 25.6
1.12 Custom IO manipulators
Example: 1
Custom IO manipulator to print integers in hexadecimal format without affecting IO global state.
#include <iostream> #include <iomanip> template<typename Numeric> struct to_hex { Numeric m_value; to_hex(Numeric value): m_value(value){ } friend std::ostream& operator<<(std::ostream& os, to_hex const& rhs) { return os << "0x" << std::uppercase << std::hex << rhs.m_value << std::nouppercase << std::dec ; } };
Usage:
>> std::cout << "x = " << to_hex<int>(2561) << " y = " << 246 << "\n"; x = 0xA01 y = 246 >> std::cout << "x = " << to_hex<int>(255) << " y = " << 246 << "\n"; x = 0xFF y = 246
Example 2 Custom IO manipulators for numerical tables.
#include <iostream> #include <iomanip> #include <type_traits> // Requires headers: <iostream>, <iomanip>, <type_traits> template< typename Numeric, // Used for restricting the types of arguments that can be used. // It will generate a compiler error for non-float arguments. typename std::enable_if<std::is_floating_point<Numeric>::value, void>::type* = nullptr > struct to_fix { Numeric m_value; size_t m_precision; size_t m_field_size; to_fix(Numeric value, size_t fieldSize = 8, size_t precision = 4) { m_value = value; m_field_size = fieldSize; m_precision = precision; } friend std::ostream& operator<<(std::ostream& os, to_fix const& rhs) { size_t prec = os.precision(); std::ios config{nullptr}; config.copyfmt(std::cout); os << std::fixed << std::setprecision(rhs.m_precision) << std::setw(rhs.m_field_size) << rhs.m_value; std::cout.copyfmt(config); return os; } };
Experiment 1:
for(double x = 0.0; x < 4.0; x += 0.5) { using T = double; double y = std::sqrt(x); std::cout << to_fix<T>(x) << to_fix<T>(y) << "\n"; }
Experiment 1 Output:
0.0000 0.0000 0.5000 0.7071 1.0000 1.0000 1.5000 1.2247 2.0000 1.4142 2.5000 1.5811 3.0000 1.7321 3.5000 1.8708
Experiment 2:
for(double x = 0.0; x < 100.0; x += 10.0) { using T = double; double y = std::sqrt(x); std::cout << to_fix<T>(x, 8, 2) << to_fix<T>(y, 9, 5) << "\n"; }
Experiment 2 output:
0.00 0.00000 10.00 3.16228 20.00 4.47214 30.00 5.47723 40.00 6.32456 50.00 7.07107 60.00 7.74597 70.00 8.36660 80.00 8.94427 90.00 9.48683
Experiment 3: Print table to file.
std::ofstream report("/tmp/report.txt"); for(double x = 0.0; x < 100.0; x += 10.0) { using T = double; double y = std::sqrt(x); report << to_fix<T>(x, 8, 2) << to_fix<T>(y, 9, 5) << "\n"; } // Sync buffer with IO // force buffer to be written to IO report.flush(); // Force close report.close(); // Run shell command to check file >> .! head -n 3 /tmp/report.txt 0.00 0.00000 10.00 3.16228 20.00 4.47214 >> >> .! tail -n 3 /tmp/report.txt 70.00 8.36660 80.00 8.94427 90.00 9.48683 >>
1.13 Redirect IO .rdbuf()
See: ios::rdbuf - C++ Reference
- Overload 1: Return pointer to stream buffer attached to stream.
- Overload 2: Set pointer to stream buffer attached to stream and clears the error state flags.
/* Overload 1 [get] */ streambuf* ios::rdbuf() const; /* Overload 2 [set] */ streambuf* ios::rdbuf (streambuf* sb);
Example: Redirect std::cerr to std::stringstream string IO
#include <iostream> #include <string> #include <sstream> std::stringstream ss; >> std::cerr << " Stderr output \n"; Stderr output >> // Backup pointer to cout stream buffer >> auto buf = std::cerr.rdbuf() (std::basic_streambuf<char, std::char_traits<char> > *) @0x7ffe4c3bc4d8 >> // Redirect stream std::cerr.rdbuf(ss.rdbuf());
The global object std::cerr prints to stringstream instead of process' stderr (standard error output):
>> std::cerr << "hello world "; >> std::cerr << "pi = " << M_PI << " exp(3.5) " << std::exp(3.5) << "\n";
Get content of stream ss:
// Stream content >> ss (std::stringstream &) @0x7fa0341bb018 >> >> ss.str() "hello world pi = 3.14159 exp(3.5) 33.1155" >>
Restore buffer:
>> std::cerr.rdbuf(buf); >> std::cerr << "pi = " << M_PI << " exp(3.5) " << std::exp(3.5) << "\n"; pi = 3.14159 exp(3.5) 33.1155
1.14 Saving and restoring IO flags
1.14.1 Manual saving restoring
IO manipulators such as std::scientific, std::fixed, std::setprecision or std::uppercase are stateful and permanently affects the IO flags what can make the code hard to understand and cause unintended IO changes. This issue can be solved by saving and restoring the IO flags.
Save IO state:
#include <iostream> #include <iomanip> // ========= Save IO flags ========= std::ios config{nullptr}; config.copyfmt(std::cout);
Before changing IO flags:
>> std::cout << "pi = " << M_PI << " gravity = " << 9.8145723e3 << " E = " << 956e6 << "\n"; pi = 3.14159 gravity = 9814.57 E = 9.56e+08 >> >> std::cout << " true = " << true << " false = " << false << "\n"; true = 1 false = 0 >>
Change IO flags:
std::cout << std::boolalpha; std::cout << std::setprecision(3) << std::fixed;
After changing IO flags:
>> std::cout << "pi = " << M_PI << " gravity = " << 9.8145723e3 << " E = " << 956e6 << "\n"; pi = 3.142 gravity = 9814.572 E = 956000000.000 >> std::cout << "sqrt(2) = " << std::sqrt(2.0) << "\n" ; sqrt(2) = 1.414 >> >> std::cout << " true = " << true << " false = " << false << "\n"; true = true false = false >>
Restore IO flags:
std::cout.copyfmt(config);
After restoring IO flags:
>> std::cout << "pi = " << M_PI << " gravity = " << 9.8145723e3 << " E = " << 956e6 << "\n"; pi = 3.14159 gravity = 9814.57 E = 9.56e+08 >> std::cout << "sqrt(2) = " << std::sqrt(2.0) << "\n" ; sqrt(2) = 1.41421 >> std::cout << " true = " << true << " false = " << false << "\n"; true = 1 false = 0
1.14.2 Automatic saving and restoring IO flags with RAAI
Class for saving IO flags using RAAI.
struct IOContext { std::ios m_config{nullptr}; std::ostream* m_pos; // Save context IOContext(std::ostream& os = std::cout): m_pos(&os) { std::cerr << " [INFO] saved IO flags" << std::endl; m_config.copyfmt(os); } // Restore context ~IOContext() { std::cerr << " [INFO] restored IO flags" << std::endl; m_pos->copyfmt(m_config); } };
Test function:
void testIOFlags() { std::cout << "===== Before =======" << "\n"; std::cout << "pi = " << M_PI << " gravity = " << 9.8145723e3 << " E = " << 956e6 << "\n"; std::cout << " true = " << true << " false = " << false << "\n"; std::cout << "===== Context - IO flags changed =======" << "\n"; { /* Sub-scope start */ IOContext ctx{std::cout}; /* Save IO Flags */ std::cout << std::boolalpha; std::cout << std::setprecision(3) << std::fixed; std::cout << "pi = " << M_PI << " gravity = " << 9.8145723e3 << " E = " << 956e6 << "\n"; std::cout << " true = " << true << " false = " << false << "\n"; /* End of sub-scope */ } // Object ctx destroyed here => IO flags restored std::cout << "===== After => IO flags restored =======" << "\n"; std::cout << "pi = " << M_PI << " gravity = " << 9.8145723e3 << " E = " << 956e6 << "\n"; std::cout << " true = " << true << " false = " << false << "\n"; } /** --- End of function testIOFlags --- */
Test in REPL:
>> testIOFlags() ===== Before ======= pi = 3.14159 gravity = 9814.57 E = 9.56e+08 true = 1 false = 0 ===== Context - IO flags changed ======= [INFO] saved IO flags pi = 3.142 gravity = 9814.572 E = 956000000.000 true = true false = false [INFO] restored IO flags ===== After => IO flags restored ======= pi = 3.14159 gravity = 9814.57 E = 9.56e+08 true = 1 false = 0 >> >>