Tools to move data from/to "TANKs "
systems
Our "TANKs" system ( tank-Mc, tank-Nt, Tank-SRT ) are connect
to a 10 Gbit/sec network (jumbo frames). Raid disks
can work at a speed in the range 500-1000 MByte/sec.
We support different tools and protocols to optimize the copy of the
files You can choise the best solution, depending on your
network connection, to optimize the work.
Please remember that the SSH based commands ( sftp, scp etc ) are
not optimized for the transfer of big files.
TCP based tools:
-- FTP: For bidirectional data transfer. ( http://www.faqs.org/docs/Linux-mini/FTP.html
)
The "TANKs" run a ftp server. Ftp clients are standard
in many distributions, or you can install from a repository.
To start ftp from a client you can use the command
> ftp tank-mc.ira.inaf.it
Our test: speed on LAN 79 MByte/s
-- WGET To download files. ( http://www.gnu.org/software/wget/manual/wget.html)
Wget clients are standard in many distribuitions.
Wget use the ftp server on the "TANKs", username and password must
be provided in line mode
>wget ftp://evlbi:PASSWORD@tank-mc.ira.inaf.it/Mc/FILE_NAME
Our test: speed on LAN 79 MByte/s
The TCP based command can give problems in the data tranfer between
far nodes (continental path) where there are congestions
on routers.
The follow commands are optimized to move data on these longh
paths.
-- GridFTP : For bidirectional data transfer ( https://wiki.birncommunity.org/display/BIRNDOC/GridFTP+Install+Guide
)
The "TANKs" run a GridFTP server that work
on IP PORT: 24383. The Grid-FTP client is not part
of a standard linux distribution, but must be installed on the
client computer.. More Info about
Grid-FTP.
> globus-url-copy
-vb ftp://evlbi:PASSWORD@tank-mc.ira.inaf.it:24383/FILE_NAME \
file:FILE.LOC
Our test:
speed on LAN 50 MByte/s
--TSUNAMI : for bidirectional transfer of vlbi data. (
http://tsunami-udp.sourceforge.net/ )
The "TANKs" host a copy of the tsunami software
that can run as a server. A copy of tsunamy software must be
installed also on the client. Tsunami can be optimize
for transfer with the parameters
You can connect (ssh ) on the "TANKs' , change directory ( cd)
where are the data thet you want to transfer and start the server
with the the command:
> tsunamid
Then on your client you must start tsunami, connect with the TANK
, sets the parameters and get/put the file.
> tsunami
tsunami> connect
tank-mc.ira.inaf.it
tsunami> set
rate 1G
tsunami> get
MYFILE
More info about the use of tsunami in the note
Policy : Data older than 60 days will be moved in the
../quarantine/ directory where they will remain for another 30
days, then will be deleted