#1 2019-06-09 15:23

therube
Member
Registered: 2010-05-24
Posts: 31

RFE: Investigate New Hash Methods

RFE: Investigate New (non-cryptographic) Hash algorithms, be it xxHash or some other.

Was thinking more along the lines of (local) duplicate file checking (shift or control open opening/adding to Hasher), where xxHash (or other) should be faster compared to MD5/SHA1...


https://cyan4973.github.io/xxHash/
https://www.voidtools.com/forum/viewtop … f=4&t=8192

Offline

#2 2019-06-10 09:21

den4b
Administrator
From: den4b.com
Registered: 2006-04-06
Posts: 3,367

Re: RFE: Investigate New Hash Methods

It is possible to add xxHash to the list of supported hashing algorithms.

However, the benefit of this algorithm appears to be quite an edge case where the content to be hashed is being read from a storage at a speed greater than 5 GB/s, which is far above the limitations of the read speeds of common consumer file storage devices.

Conventional spinning disks can reach 200 MB/s, SATA based SSD drives can reach 500 MB/s, and even the top of the range Samsung NVMe SSD 970 PRO can barely reach 3GB/s in practice.

Do you have a scenario or a setup where you can extract value out of using xxHash algorithm?

Offline

Board footer

Powered by FluxBB