Every file format is governed by a specification—a standardized structure that dictates how the bytes within a file are arranged. Much like how internet protocols have strict standards, file types also follow agreed-upon rules. When an application opens a file such as a PDF or reads a PNG image, it interprets the data by adhering to these predefined guidelines. While we often rely on file extensions like .zip, .txt, or .jpg to identify a file's type, these extensions serve primarily as labels for human convenience and operating system recognition rather than definitive identifiers. This is why simply renaming a file from photo.jpg to photo.png does not convert the image format; the underlying data remains unchanged. The true method for determining a file’s format lies in what is called a "magic number." A magic number is a distinctive sequence of bytes located at the start or at specific positions within a file, acting as a unique identifier for its format. Each file format has an internationally recognized magic number that applications check to confirm the type of file they are handling, regardless of what the file extension indicates. For instance, PNG files begin with the byte sequence 89 50 4E 47, while ZIP files start with 50 4B 03 04. Bitmap images are identified by the first two bytes 42 4D, which corresponds to "BM" in ASCII, short for bitmap. To illustrate, consider a simple Go program that opens a file with a .bmp extension and reads the first two bytes to verify whether it truly is a bitmap image. The program compares these bytes to the expected BMP signature. If the bytes match, it confirms that the file is a valid BMP; otherwise, it flags the file format as invalid. This approach is similar to the Unix "file" command, which relies on such signatures to identify files irrespective of their extensions. Beyond the magic number, many file formats include metadata following the initial signature. This metadata varies by format but can include details such as image dimensions, audio sample rates, or document author information. Understanding these elements is crucial for applications that need to process or manipulate the file content effectively. File formats typically fall into three broad structural categories. The first category consists of binary formats with a rigid structure, such as PNG, JPEG, and MP3 files. In these formats, every byte position has a specific meaning defined in the specification, and programs parse them by reading exact byte offsets. The second category is text-based structured formats like JSON, XML, HTML, and CSV. These are human-readable and follow grammatical rules, making them easier to debug but often larger in size. The last category includes container formats such as ZIP, MP4, and PDF. These are more complex, acting as filesystems within a file, and can contain multiple embedded files or data streams. For example, an MP4 file might contain separate tracks for video, audio, and subtitles, while a DOCX file is essentially a ZIP archive containing XML documents. With an understanding of file format specifications, magic numbers, and data structures, it is possible to write custom parsers to read and manipulate a wide variety of file types. This foundational knowledge enables developers to perform tasks such as converting images to greyscale by directly modifying the byte content of the files, a technique that will be explored in future work.
