How to copy data from/to ZFS filesystem
The diagram above shows how the architecture how stash is connected with cluster nodes.
From Lustre to ZFS
To copy/move data from Lustre to ZFS we strongly recommend to use rsync. The best way to move data is through the infiniband network, in order to take advantage of the bandwith.
In the following example, we are going to copy 3 files (100G, 10G and 1G) from the directory /.lustre/aoc/sciops/lsaavedr/zfs/ to my zfs area /stash/users/lsaavedr/FromLustre. In this example, I am going to use nmpost039 to take advantage of the Infiniband network.
[lsaavedr@nmpost039 ~]$ mkdir /stash/users/lsaavedr/FromLustre [lsaavedr@nmpost039 ~]$ ls -hl /.lustre/aoc/sciops/lsaavedr/zfs/ total 109G -rw-r----- 1 lsaavedr nmstaff 98G Sep 23 13:55 test01-lustre-s100G.img -rw-r----- 1 lsaavedr nmstaff 9.8G Sep 23 13:53 test01-lustre-s10G.img -rw-r----- 1 lsaavedr nmstaff 1000M Sep 23 13:53 test01-lustre-s1G.img [lsaavedr@nmpost039 ~]$ rsync -avh /.lustre/aoc/sciops/lsaavedr/zfs/test01-lustre-s1* /stash/users/lsaavedr/FromLustre sending incremental file list test01-lustre-s100G.img test01-lustre-s10G.img test01-lustre-s1G.img sent 116.42G bytes received 73 bytes 275.55M bytes/sec total size is 116.39G speedup is 1.00
From ZFS to Lustre
In the following example, we are going to copy 3 files (100G, 10G and 1G) from the directory /stash/users/lsaavedr/lustre/ to my lustre area /.lustre/aoc/sciops/lsaavedr/FromZFS. In this example, I am going to use nmpost039 to take advantage of the Infiniband network.
[lsaavedr@nmpost039 ~]$ mkdir /.lustre/aoc/sciops/lsaavedr/FromZFS [lsaavedr@nmpost039 ~]$ ls -l /stash/users/lsaavedr/lustre/test01-lustre-s1* -rw-r----- 1 lsaavedr nmstaff 104857600000 Sep 23 13:55 /stash/users/lsaavedr/lustre/test01-lustre-s100G.img -rw-r----- 1 lsaavedr nmstaff 10485760000 Sep 23 13:53 /stash/users/lsaavedr/lustre/test01-lustre-s10G.img -rw-r----- 1 lsaavedr nmstaff 1048576000 Sep 23 13:53 /stash/users/lsaavedr/lustre/test01-lustre-s1G.img [lsaavedr@nmpost039 ~]$ rsync -avh /stash/users/lsaavedr/lustre/test01* /.lustre/aoc/sciops/lsaavedr/FromZFS sending incremental file list test01-lustre-s100G.img test01-lustre-s10G.img test01-lustre-s1G.img sent 116.42G bytes received 73 bytes 295.11M bytes/sec total size is 116.39G speedup is 1.00
