This page describes, in Unix manual page style,
a program
available for downloading from this site
which copies data while enforcing a specified maximum transfer
rate in bytes per second by pausing between data blocks. This
permits bulk data transfers without "locking up" disc or
network bandwidth to the detriment of other applications and/or
users. It can be thought of as the equivalent of the
Unix nice command for I/O.
NAME
valve - copy data, enforcing transfer rate limitation
SYNOPSIS
valve
[
options
]
[
infile
[
outfile
] ]
DESCRIPTION
Usually, when you copy one file to another or transfer data between
applications, you want things to finish as soon as possible--why wait
any longer than necessary? However, when the operation in question
consumes scarce shared disc or network transfer bandwidth (for
example, when copying large files between machines on a local network
or across the Internet), it may be wise to limit the transfer rate to
avoid degrading performance for other users or applications. In an
ideal world, wise and benevolent operating systems would
automatically throttle bandwidth to achieve this goal, but as we all
know, ours is not an ideal world, nor are operating systems, for the
most part, wise or benevolent.
valve
is a Unix/Linux/etc. command line utility which copies an input
source to an output destination, pausing as necessary between
blocks of data to achieve a requested mean transfer rate. Options
permit measuring transfer rate without enforcing a limit and
determining the timing granularity of the system on which the program
is run.
OPTIONS
Long
(
--)
options may be abbreviated to their shortest unique prefix
and single-letter options without arguments may be
aggregated. Arguments to single-letter options may immediately
follow the option or be separated by white space.
- --alwayswait, -a
-
If the measured mean transfer rate has fallen sufficiently
behind the requested rate such that no pause between blocks is
indicated,
valve
normally skips the pause. If this option is set, a minimum-length
pause will always occur after each block copied. This option can
avoid pathological behaviour on copies with an inconsistent
transfer rate.
- --blocksize, -b size
-
Data will be copied in blocks of
size
bytes. The
size
argument may contain commas and include a size factor suffix as
described below. If no
--blocksize
is specified, a block size will be chosen such that four blocks
per second are copied at the requested transfer
--rate,
with an upper limit on block size of one megabyte.
- --copyright
-
Print program copyright information.
- --help, -u
-
Print how-to-call information.
- --measure, -m
-
Measure, but do not limit, the transfer rate and print it
at the end of the copy. Note that the
--blocksize
setting may influence the transfer rate.
- --rate, -r bytesec
-
The transfer rate will be limited to
bytesec
bytes per second. The argument may contain commas and include a
size factor suffix as described below. If no
--rate
is specified, a rate of one megabyte per second will be used.
- --summary, -s
-
Print a summary of the number of bytes copied and the
actual transfer rate at the end of the copy.
- --timingtest secs
-
Run a timing test for
secs
seconds to determine the minimum wait interval supported
by the operating system on the machine running the test.
Knowing this value permits skipping pauses which would
inadvertently reduce the transfer rate below the desired
value. (See the
--alwayswait
option.) The timing test shows the default minimum
wait time built into
valve
and the measured minimum interval, both in nanoseconds.
If there is a radical difference between these values
(there's no need to worry about a factor of three or less),
you may wish to specify the actual value with the
--waitmin
option or rebuild the program, configuring it with the
MIN_SLEEP_NSEC=
variable set to the measured value.
- --version
-
Print program version information.
- --waitmin nsecs
-
Pauses shorter than
nsecs
nanoseconds will be skipped unless the
--alwayswait
option is specified.
The
waitmin
argument may contain commas and include a size factor suffix as
described below. If no
--waitmin
is specified, the value built into the program with the
MIN_SLEEP_NSEC=
configuration variable will be used, with a default of
``10,000,000''
(10 milliseconds).
SCALE FACTOR SUFFIXES
Arguments to the
--blocksize,
--rate, and
--waitmin
options can include commas to separate groups of digits, and
may be followed by a suffix indicating a scale factor for
the numeric argument as follows. Note that these suffix
specifications are case-sensitive.
| c | character | 1
|
| w | word | 2
|
| b | block | 512
|
| KB, kB | kilo | 1,000
|
| K, k | kibi | 1,024
|
| MB | mega | 1,000,000
|
| M | mebi | 1,048,576
|
| GB | giga | 1,000,000,000
|
| G | gibi | 1,073,741,824
|
| TB | tera | 1,000,000,000,000
|
| T | tebi | 1,099,511,627,776
|
| PB | peta | 1,000,000,000,000,000
|
| P | pebi | 1,125,899,906,842,624
|
| EB | exa | 1,000,000,000,000,000,000
|
| E | exbi | 1,152,921,504,606,846,976
|
| ZB | zetta | 1,000,000,000,000,000,000,000
|
| Z | zebi | 1,180,591,620,717,411,303,424
|
| YB | yotta | 1,000,000,000,000,000,000,000,000
|
| Y | yobi | 1,208,925,819,614,629,174,706,176
|
A trailing
``D''
may be used to denote a decimal power of ten instead
of
``B''.
EXIT STATUS
valve
returns status 0 if processing was completed without errors, 1 if an
I/O or internal error occurred, and 2 if processing could not be
performed at all due, for example, to a nonexistent input file
or command line syntax error.
EXAMPLES
Suppose you're logged into a local system and you with
to transfer the contents of the current directory to
a directory named
``/mirror/files/
''
on a remote system named
somewhere.ratburger.net,
while limiting the transfer rate to 256 Kb/sec to avoid
saturating your outbound bandwidth or the inbound bandwidth of the
destination site. You can accomplish this with:
tar cf - . | valve --rate 256K | \
ssh somewhere.ratburger.net "(cd /mirror/files; tar xf -)"
FILES
If no
infile
is specified or
infile
is a single ``-'',
valve
reads from standard input; if no
outfile
is given, or
outfile
is a single ``-'',
output is sent to standard output. The input and
output are processed strictly serially; consequently
valve
may (and usually will) be used in pipelines.
The program
can process files of any size supported by the system with any
binary content whatsoever.
BUGS
This is a Unix application through and through, and will
require substantial work to port to other operating systems,
particularly in the area of high-precision waits. It does,
however, work on 32-bit Windows systems with
Cygwin installed.
valve
uses the
nanosleep
function, if available, to pause between blocks of data.
If the system does not support that function,
usleep
is used, if available, and failing that, an emulation
of
usleep
with
setitimer
and
signal.
If none of these facilities are available, you're out
of luck.
It is tacky using
``B''
to denote a decimal power of ten in scale factor suffixes,
but that's how
dd
does it, so I opted in favour of compatibility. You can use a
trailing
``D'',
if you prefer, to indicate a decimal power.
Please report bugs and documentation errors to
Bravo Uniform Golf Sierra @ Foxtrot Oscar Uniform Romeo Mike India
Lima Alpha Bravo Decimal Charlie Hotel. If you don't have any idea
what I just said, please
consult
this document.
SEE ALSO
cat(1),
dd(1),
nanosleep(2),
setitimer(2),
signal(2),
usleep(3)
valve C source code: valve-1.0.tar.gz
Read valve source code (requires
Acrobat Reader)
AUTHOR
John Walker
http://www.fourmilab.ch/
This software is in the public domain. Permission to use, copy,
modify, and distribute this software and its documentation for
any purpose and without fee is hereby granted, without any
conditions or restrictions. This software is provided "as is"
without express or implied warranty.
Download