Spending Hours Automating a Task That Takes a Few Minutes

The cover image for this post is not a joke. It is based on reality.

On today’s episode of things that took way too much time, I used rust to automate a simple task. For context on what the task is, I’m a massive doom fan. I enjoy the latest entries in the series. However, the classic doom games are my most played even to this day.

In the late 90s to early 2000s, my father played this game frequently. I remember walking by the room where he would play, and hearing the terrifying noises of the zombies and demons. It was quite scary being that young. The more I watched him play it, the less scared I was of the game. I got to the point where I wanted to play it myself. One day, my father set me up on one his old trusty thinkpad laptops. He launched doom and let me take control. I remember getting my ass kicked. Frustrated and angry, my father advised me to type iddqd. He said it would help me beat the game. And sure enough, it did. I played through the entire campaign over the course of the next few months.

After beating the doom 2 campaign, I thought that was it. But then my father introduced me to the world of custom wads. He showed me a whole new world of custom content on the website Doomworld Forums. I couldn’t get enough. There’s so much custom content out there for doom, that it’s infinitely replayable. To this day, I play the classic doom titles. My main gameplay mod of choice is Project Brutality.

The task at hand⌗

I use the source port GZDoom to play the classic titles. The source port adds opengl and vulkan. This allows true 3D free-look, and other graphics enhancements. I follow the development of the source port on github. There exists a group of developers within the doom community known as The DRD Team. They have an SVN Repository for GZDoom. The SVN Repository hosts the most bleeding edge builds of GZDoom that include experimental features.

I use a graphical program called ZDL to pick and choose what classic doom title to play, as well as my source port of choice. The manual steps for setting up gzdoom:

1. Download the latest GZDoom release from the SVN Repository.
2. Use 7zip to extract the archive.
3. Launch ZDL. Click General Settings, and click on the ➕ icon. A file explorer window appears prompting you to browse for where the extracted gzdoom executable resides.
4. Click the ➕ icon on the IWADS menu and navigating to the directory where your doom IWADS are stored.
5. Save the configuration as an ini file. Or overwrite the older one if it exists.

I wanted a program to do all of that for me because I am that lazy.

Why Rust 🦀?⌗

My original intent was to use python 🐍 for this task. I wrote something very quickly because Python is awesome for whipping up programs for these kinds of things. However, the limitation came when it was time to extract the downloaded 7zip archive. I used py7zr to handle extracting the 7zip files. It seemed promising, except for the fact the downloaded 7zip archives were made using the BCJ2 preprocessor. The py7zr package is incompatible with BCJ2:

File "C:\Users\jmaguy\Downloads\ZDL_3-1.1_Win_x86\venv\lib\site-packages\py7zr\compressor.py", line 1158, in raise_unsupported_method_id
    "BCJ2 filter is not supported by py7zr."
py7zr.exceptions.UnsupportedCompressionMethodError: (b'\x03\x03\x01\x1b', 'BCJ2 filter is not supported by py7zr. Please consider to contribute to XZ/liblzma project and help Python core team implementing it. Or please use another tool to extract it.')

More research into related python packages that can extract 7zip packages was not looking good. I started to think about using another tool to get the job done.

For a brief moment I considered using C++. It had a pretty promising library called bit7z. Frankly, dependency management sucks in C++. I didn’t not feel like setting up CMake, or spend countless hours dicking around with library paths in Visual Studio. C# crossed my mind too. But when I discovered the NuGet package used to extract 7zip files requires you to buy it:

I ended up choosing Rust because it has the dependency management convenience of python, and has a plethora of third party addons like C++. To be fair, Python also has a plethora of addons, but for whatever reason, Guido does not like 7zip files.

More Effort Than What It Was Worth⌗

I created a new cargo project. Used the following dependencies:

[package]
name = "gzdoom_svn_downloader"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
reqwest = { version = "0.11", features = ["blocking"] }
scraper = "0.13.0"
dirs = "4.0"
compress-tools = "0.13.0"
rust-ini = "0.18.0"

Compress tools was a lot of trouble to get working on windows. I used chocolatey to install rust. Chocolatey has two different versions of rust available. You can choose Rust (GNU ABI), or you can use Rust (MSVS ABI). The version using the GNU ABU requires you to use pkg-config to point rust to the appropriate libraries it needs to perform the 7zip decompression steps. From my experience, this tool on windows was difficult and I couldn’t get it working. I ended up using the Rust MSVS ABI version, which had a better library management that I will get to in a minute.

Note to self: Next time use rustup, the official rusty way of installing rust

Installing libarchive⌗

This is a library that needs to be linked to rust for compression tools to work. The crate documentation recommends using vcpkg to install it on windows. Vcpkg is a free package manager for C/C++ to fix the terrible package management problem. The getting started guide goes over installing vcpkg:

cd C:\src\
git clone https://github.com/Microsoft/vcpkg.git
.\vcpkg\bootstrap-vcpkg.bat
cd .\vcpkg
.\vcpkg.exe install libarchive:x64-windows-static-md

Please note i’m using the triplet x64-windows-static-md because by default vcpkg was installing libarchive to a triplet not recognized by rust. Explicitly setting the triplet like this fixed the issue.

At this point, I’m already frustrated and realized that I spent way too much time on this. But the sunken cost fallacy was kicking in, and I pushed forward.

Rust⌗

The complete code for my implementation.

use std::io::Read;
use dirs::home_dir;
use std::path::Path;
use std::fs::File;
use std::io;
use compress_tools::*;
use std::result::Result;
use ini::Ini;
use scraper::{Html, Selector};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let base_url = "https://devbuilds.drdteam.org";
    // $HOME/Downloads on UNIX, C:\users\<USERNAME>\Downloads on Windows
    let download_directory = Path::new(".").join(home_dir().unwrap()).join("Downloads");

    // Make a request to the SVN Server to get list of builds
    let mut res = reqwest::blocking::get(base_url.to_owned() + "/gzdoom/")?;
    let mut body = String::new();
    res.read_to_string(&mut body)?;

    // Use Scraper to parse body
    let document = Html::parse_document(&body);
    let table_selector = Selector::parse("table").unwrap();
    let tr_selector = Selector::parse("tr").unwrap();
    let a_selector = Selector::parse("a").unwrap();

    // select table of latest builds available
    let table = document.select(&table_selector).next().unwrap();

    // Grab the latest build available, aka the first element in the table
    let latest_gzdoom_url = table.select(&tr_selector).collect::<Vec<_>>()[1]
        .select(&a_selector).next().unwrap().value().attr("href").unwrap();
    let latest_gzdoom_url = &(base_url.to_owned() + latest_gzdoom_url);

    // download the file from url
    let latest_filename = download_directory.join(Path::new(latest_gzdoom_url).file_name().unwrap().to_str().unwrap());
    res = reqwest::blocking::get(latest_gzdoom_url)?;
    let mut out = File::create(latest_filename.to_str().unwrap()).expect("failed to create file");
    io::copy(&mut res, &mut out)?;

    // Extract file
    let directory_name = Path::new(latest_gzdoom_url).file_stem().unwrap().to_str().unwrap();
    let extracted_directory = download_directory.join(directory_name);
    let mut source = File::open(latest_filename)?;
    compress_tools::uncompress_archive(&mut source, &extracted_directory, Ownership::Preserve)?;

    //Update INI FILE
    let zdl_config_ini = download_directory.join("ZDL_3-1.1_Win_x86").join("my_config.ini");
    let mut conf = Ini::load_from_file(&zdl_config_ini).unwrap();
    conf.with_section(Some("zdl.ports"))
        .set("p0n", directory_name)
        .set("p0f", extracted_directory.join("gzdoom.exe").to_str().unwrap());
    conf.write_to_file(zdl_config_ini.clone()).unwrap();

    Ok(())
}

Rust has an enum called Option that is used when absence is a possibility. From my understanding unwrap() implicitly will return the inner element, or trigger a panic. The Option gives us meaningful control over error handling and allows us to define explicit handling. Theres a lot of instances of unwrap() used within my code, daisy chained together. I had to use this unwrap method call to convert types. For example:

// Grab the latest build available, aka the first element in the table
let latest_gzdoom_url = table.select(&tr_selector).collect::<Vec<_>>()[1]
    .select(&a_selector).next().unwrap().value().attr("href").unwrap();

It looks really messy and awkward in my opinion. I’m sure I’m not adhering to the Rust way of doing things here. If there are any cringing Rust experts here, I’d really like your input. The collect method is gathering each table tr entry into a Vector container, although I’m only concerned with the first element. The next method is called to advance the iterator and return an Option enum that contains an a html element. As mentioned before unwrap will return the tr element itself from within the option. value() will return a reference of the a element. The attr method selects the href link to the latest svn build. Finally we throw one more unwrap in there to return a string from the option. A whole lot of chaining and type conversions is what it seems. Just seems bizarre as someone new to Rust. But again to be fair, there’s most likely a much better solution to this that I don’t know about.

All said and done, the program accomplishes what i set out to do. I now have button press to grab the latest GZDoom build and update my ZDL configuration file. This allows me to jump right into gaming 🎮. Playing doom will help me alleviate the anger and frustration.