scripts that NVIDIA distributed to clients so they could automatically download and preprocess The Pile dataset.

sounds like they allegedly wrote some stuff to get faster downloads/avoid throttling while they were allegedly pirating books from shadow libraries for their AI