Linux Split & Join Large File

I had a big file to send over an unreliable/choppy internet connection with no resume function. The only solution I could think of is to split it into multiple files so that if disconnected I do not need to restart from the beginning.

Split

Assume the large file is 2G and I need to split it into multiple files of 700M.

split -b 700M large.dat
# this will produce
# xaa 700M
# xab 700M
# xac 600M

You can also customize the output filenames

split -b 700M large.dat small.dat.x
# this will produce
# small.dat.xaa
# small.dat.xab
# small.dat.xac

Below are extracted from man split. Abbreviation for sizes are

The  SIZE  argument  is  an  integer  and  optional  unit  (example: 10K is 10*1024).  Units are
K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (powers of 1000).

Join

Assume my output files are xaa, xab & xac. To join the file

cat x* > large.dat

The above command works if your Linux locale is English. If unsure run the below command to be safe else it might join in the wrong sequence.

cat xa{a..c} > large.dat

< Back to all the stories I had written