this post was submitted on 21 Apr 2025
16 points (90.0% liked)

Programming

19722 readers
71 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 2 years ago
MODERATORS
 

https://github.com/thingsiplay/crc32sum

# usage: crc32sum [-h] [-r] [-i] [-u] [--version] [path ...]

crc32sum *.sfc
2d206bf7  Chrono Trigger (USA).sfc

Previously I used a Bash script to filter out the checksum from 7z output. That felt always a bit hacky and the output was not very flexible. Plus the Python script does not rely on any external module or program too. Also the underlying 7z program call would automatically search for all files in sub directories recursively when a directory was given as input. This would require some additional rework, but I decided it is a better idea to start from scratch in a programming language. So I finally wrote this, to have a bit better control. My previous Bash script can be found here, in case you are curious: https://gist.github.com/thingsiplay/5f07e82ec4138581c6802907c74d4759

BTW, believe it or not, the Bash script running multiple commands starts and executes faster than the Python instance. But the difference is negligible, and the programmable control in Python is much more important to me.


What is this program for?

Calculates the CRC hash for each given file, using Python's integrated zlib module. It has a similar use like MD5 or SHA, but is way, way weaker and simpler. It's a quick and easy method to verify the integrity of files, in example after downloading from the web, to check data corruption from your external drives or when creating expected files.

It is important to know and understand that CRC-32 is not secure and should never be used cryptographically. It's use is limited for very simple use cases.

Linux does not have a standard program to calculate the CRC. This is a very simple program to have a similar output like md5sum offers by default. Why use CRC at all? Usually and most of the time CRC is not required to be used. In fact, I favor MD5 or SHA when possible. But sometimes, only a CRC is provided (often used by the retro emulation gaming scene). Theoretically CRC should also be faster than the other methods, but no performance comparison has been made (frankly the difference doesn't matter to me).

you are viewing a single comment's thread
view the rest of the comments
[–] FizzyOrange@programming.dev 3 points 3 days ago (1 children)

I guess you could say "the name"!

In fairness the first and second results on Google point to the crc32 tool...

https://askubuntu.com/a/303666

$ sudo apt-get install libarchive-zip-perl
$ crc32 my_file

Again not a great package name and it does require Perl, but in Linux at least that's a less painful dependency than Python.

[–] thingsiplay@beehaw.org 2 points 3 days ago* (last edited 3 days ago) (1 children)

I probably am bad at web searching (don't do this long enough and don't use Google either). At least this was a learning experience for myself, so its not totally waste of time for me. And maybe someone else can look in the code and learn. Now I see there are alternatives available, such as

pacman -Si python-crc32c
[–] FizzyOrange@programming.dev 4 points 3 days ago (1 children)

Yeah good learning experience. We've all created projects and then found they already exist.

[–] sik0fewl@lemmy.ca 5 points 3 days ago

Sometimes the only way to find the tool you're looking for is to create it yourself, share it on forums and wait for people to berate you and ask why you wouldn't just use X.

It's a time-honoured tradition.