Network
The river network a simulation routes over: identity, topology, and Muskingum parameters read from a network
file, plus the concurrent routing partition, the signs of the routing coefficients derived from them, the sub-reaches
and subcycles that use_subreaches and use_subcycles route to make every coefficient non-negative, and the stabilized network that writes
those sub-reaches as rows. A Router builds one from its Configs and reuses it, so a network table is parsed and
partitioned once no matter how many simulations run over it.
Network
A wrapper around the network table, the dataframe that describes the number, topology, and DFS order of the rivers and their Muskingum parameters.
When instantiating this class, the network file is eagerly read and the connectivity is set. This may make instantiation have a small but non-negligible time cost.
size
property
The number of rivers, one per row of the network table.
river_ids
property
(n,) id of each river, in the order of the network table.
next_river_ids
property
(n,) id of the river each river drains into, -1 at a basin outlet.
k
property
(n,) Muskingum k of each river, its travel time in seconds.
x
property
(n,) Muskingum x of each river, its weighting of inflow against outflow, from 0 to 0.5.
upstream_counts
property
(n,) number of rivers upstream of each river, which in DFS order are the rows immediately before it.
synthetic
property
(n,) True for each sub-reach stabilize added, or None on a network that was never stabilized.
parent_river_ids
property
(n,) id of the original river each row was split from on a stabilized network, or None.
original_river_ids
property
Id of each river that is not a synthetic sub-reach, in order: the rows of every routed discharge array.
__init__(network_file)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network_file
|
PathInput | DataFrame
|
the network file parquet, or the network table itself as a DataFrame, which is copied |
required |
from_configs(configs)
classmethod
Build a Network from the network_file of a Configs object, its k and x routed exactly as given.
check_parameters()
Check that k and x are in expected, valid ranges. np.isfinite checks for NaN and inf
routing_blocks(threads=1)
The blocks of rivers each job routes, packed for threads jobs by streams.assign_blocks. They are
derived once per thread count and cached, so repeated simulations over this network never rebuild them.
Each block is a contiguous range of rows read from the upstreamCount column, so nothing is reordered: the forcing, the state, and the routed discharge stay in network file order from end to end. The best blocks depend on the thread count, so they are rebuilt for each one. On one thread, the main stem is the whole region.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
threads
|
int
|
number of jobs the sub-watershed blocks are packed into |
1
|
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
tuple[JobBlocks, ...]
|
the (block_starts, block_stops) of each job, and last the main stem's, which routes after them. |
validity_window()
The inclusive range of routing time steps over which every Muskingum coefficient of each river is non-negative,
(2*k*x, 2*k*(1-x)), the steps at which its routing is valid. Outside it the scheme is still stable and
conserves mass, but its discharge can go negative.
invalid_mask(dt, layout=None)
Per-river masks of the two ways a river's coefficients go negative at dt, one step for every river or one
per river, for the whole river or, with layout, for any of its sub-reaches at its own step, dt over its
subcycles.
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
tuple[ndarray, ndarray]
|
(too_long, too_short). |
sub_reaches(dt)
The equal sub-reaches of each river at dt, one step for every river or one per river: the fewest
sub-reaches, laid out as conditioning lays out equal ones, whose coefficients are all non-negative at
dt, which is what use_subreaches routes for rivers too long for their step. An equal sub-reach of travel
time k/N has a non-negative c1 when k/N <= dt + (1-2x)*k, and a sub-reach whose x is 0 has one at any
step, so the count is ceil(k / (dt + (1-2x)*k)): 2 for every river too long at x = 0.2, and never more
than ceil(1/(1-2x)). No split into fewer sub-reaches of any travel times is valid at dt.
The shortest sub-reach has a non-negative c3 when dt <= 2*k_N*(1 - x_N). A river too short for dt cannot
meet it, since its sub-reaches are shorter still, and for x <= 3/8 every river too long for dt does. A river
whose sub-reaches do not is kept as one reach and reported as unresolvable, and conditioning splits it into
unequal sub-reaches that take subcycles of their own instead, where it uses subcycles too.
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
tuple[IntArray, ndarray]
|
(sub_reaches, resolvable). |
subcycles(dt)
The subcycles of each river at dt, one step for every river or one per river: the fewest equal steps a river
is routed in within each step of dt, overriding dt with the shorter routing step dt / subcycles of its
own. Configs(dt_routing='auto') identifies each river's own valid step with them, dt_runoff /
subcycles, and use_subcycles gives them to rivers too short for a dt_routing in seconds. The count is
ceil(dt/(2*k*(1-x))): the smallest that brings the river's own step under the upper bound 2*k*(1-x).
A river that is not too short takes 1.
Its own step can also fall under the lower bound 2*k*x, when the window is narrower than one whole count of
subcycles; such a river keeps one step and is reported as unresolvable. For x <= 1/3 the window always
holds one.
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
tuple[IntArray, ndarray]
|
(subcycles, resolvable). |
conditioning(dt, *, use_subreaches=True, use_subcycles=True)
How each river is routed at dt, one step for every river or one per river, as the use_subreaches and
use_subcycles of a Configs route it: a river too long for dt is split into its equal sub_reaches with
use_subreaches, and a river too short for it is routed in its subcycles, equal steps of its own, with
use_subcycles. Without them a river is routed whole at dt. With both, a river neither makes valid, which only a
river with x > 1/3 can be, is split into the fewest sub-reaches of unequal travel times that each take
subcycles of their own and keep the river's diffusion, those of splitting.fewest_sub_reaches, and is kept
whole if no split into at most the 256 sub-reaches the kernels route one river as is valid. stabilize writes
this layout as rows.
Returns:
| Name | Type | Description |
|---|---|---|
Layout |
Layout
|
the sub-reaches of each river, and the travel time, x, and subcycles of dt of each |
check_routable(sub_reaches)
Raise ValueError unless the routing kernels can route this network as sub_reaches sub-reaches per river, as
from conditioning: no more rivers, sub-reaches, sub-reaches of one river, or inflow rivers of one river than
river_route/kernels/c/checks.c allows. Routing checks it before any runoff is read, so a network the kernels
would refuse is refused while it is prepared, with the river that breaks the limit named. A river with an x
near 1/2 is split into the most sub-reaches; see the routing algorithm docs for holding x lower.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sub_reaches
|
IntArray
|
the sub-reaches of each river, 1 for a river that is not split |
required |
check_validity(dt, action='raise', layout=None)
Report rivers with a negative Muskingum coefficient at dt and take the configured action. With layout a
river is checked as its sub-reaches, each at its own step.
Every coefficient is non-negative when 2*k*x <= dt <= 2*k*(1-x). Outside that window the scheme is still
stable and still conserves mass, since c1+c2+c3 == 1 holds for negative coefficients too, but its discharge can
dip below zero ahead of a rise (c1 < 0) or alternate from step to step and go negative (c3 < 0). The kernels
write negative discharge as routed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dt
|
float
|
the routing time step to check against |
required |
action
|
Literal['log', 'raise', 'ignore']
|
|
'raise'
|
layout
|
Layout | None
|
optional sub-reaches of each river and the subcycles of each, as from |
None
|
check_stabilizable()
Raise ValueError for rivers with an x above MAX_STABILIZED_X, which routing with use_subreaches or
use_subcycles, and stabilize, refuse. A river needs up to ceil(1/(1-2x)) sub-reaches, 50 at x = 0.49,
and many more above it, and at x = 1/2 its coefficients are all non-negative only at a step equal to its travel
time, so that it can be impossible to stabilize.
stabilize(dt, *, weights=None)
Stabilize this network in place: every river that conditioning splits at dt is replaced by its
sub-reaches in series, whose coefficients are all non-negative at dt or in the subcycles of it each takes,
which routing with use_subcycles gives each row. The original river keeps its id and becomes the outlet
sub-reach, and the added sub-reaches are injected directly upstream of it with synthetic True and ids
counting down from -1,000,000. Deleting the synthetic rows gives back the original row order. A network is
stabilized once: to stabilize it at another dt, build a new Network from the original network file.
The sub-reaches are those of conditioning, which is what use_subreaches and use_subcycles route inside the
kernel, so they keep each river's mean travel time k and its travel time variance k**2 * (1 - 2x). A river
they cannot make non-negative is kept whole. Rivers left with a negative coefficient are not reported here:
invalid_mask finds them, and routing reports them as invalid_coefficients says.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dt
|
float
|
the routing time step at which, or in subcycles of which, every coefficient of every sub-reach must be non-negative |
required |
weights
|
list | None
|
optional explicit sub-reaches, one sequence of relative lengths per river in network table
order. Each river's k is apportioned in proportion to its weights, which is what matches sub-reaches
to real segment geometry, and its sub-reaches all take |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
this network, stabilized, with a |
Raises:
| Type | Description |
|---|---|
ValueError
|
if this network is already stabilized or dt is not positive |
write_stabilized(dt, path=None, *, weights=None)
Stabilize this network for dt and write it as a network table with the synthetic column.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dt
|
float
|
the routing time step at which, or in subcycles of which, every coefficient of every sub-reach must be non-negative |
required |
path
|
PathInput | None
|
where to write the parquet. Defaults to |
None
|
weights
|
list | None
|
passed to |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
Path |
Path
|
the file written |
to_parquet(path)
Write the network table, with every column in its dtype of NETWORK_DTYPES, to a parquet file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
PathInput
|
the parquet file to write |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Path |
Path
|
the file written |
Layout
Bases: NamedTuple
How each river is routed at a step: as one or more sub-reaches in series, each river's upstream to downstream, each with its own travel time, x, and subcycles of the step. A river routed whole is one sub-reach of its own k and x.
Splitting
The search for the fewest sub-reaches of unequal travel times, each in subcycles of its own, that
Network.conditioning splits a river into where equal sub-reaches and subcycles cannot make it valid alone.
river_route.network.splitting
The fewest sub-reaches a river splits into when equal sub-reaches cannot make every coefficient of it non-negative at a step. The sub-reaches may differ in travel time, each takes subcycles of the step of its own, and every one keeps the river's diffusion: a sub-reach of travel time k_i takes x_i = 1/2 - (1/2 - x) k / k_i, which keeps the river's travel time variance k**2 (1 - 2x) across them whatever their travel times. The window of steps at which a sub-reach is valid, 2 k_i x_i <= step <= 2 k_i (1 - x_i), is then |step - k_i| <= (1 - 2x) k, as wide for every sub-reach of the river.
In m subcycles of a step dt, a sub-reach may take any travel time within (1 - 2x) k of dt/m and no shorter than (1 - 2x) k, where its x is 0, so the travel times a sub-reach may take are a union of windows about dt, dt/2, dt/3, and so on. A river splits into n sub-reaches when its k is a sum of n of them. The totals of 1, 2, 3, ... sub-reaches are grown as sorted intervals, and n sub-reaches make k when a total of the first n - n//2 of them and a total of the last n//2 do, so the totals are grown only to half the fewest n. The window of each sub-reach is then read back from them.
fewest_sub_reaches(k, x, dt, most)
The travel times, upstream to downstream, of the fewest sub-reaches of a river, at most most, whose
coefficients are all non-negative at dt, and the subcycles of dt each takes: the fewest whose step is in its
window, as Network.subcycles gives a river. Empty arrays where no split of at most most sub-reaches is valid,
as at x = 1/2, whose windows close to single steps.
split_rivers(k, x, dt, most)
The fewest_sub_reaches of each river.
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
tuple[IntArray, Float64Array, IntArray]
|
(sub_reaches, travel_times, subcycles). |
Streams
The functions a Network partitions its rivers with, the analysis of how evenly a partition divides a network,
and subsetting a network file, and its grid weights, to one river and every river upstream of it.
river_route.network.streams
Functions of a network table in DFS order: the sub-watershed blocks threads route and how evenly they divide a network, and the subset of a network file, and of its grid weights, to one river and every river upstream of it. The blocks and the subset only read the contiguous run of rows that a river's riverIndex and upstreamCount give.
assign_depth_first_search_order(df)
Assigns a Depth First Search (DFS) order and upstream count to a DataFrame representing a river network: the rows reordered so that every river follows the rivers upstream of it, and the riverIndex and upstreamCount columns that describe that order. Rivers draining into the same river, and basin outlets, keep their relative order, so a table already in DFS order keeps its order. The DataFrame given is not changed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
the network table, with a riverId column and a nextRiverId column that is -1 at an outlet |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
pd.DataFrame: df copy with new riverIndex and upstreamCount columns ascending sorted by the new riverIndex |
Raises:
| Type | Description |
|---|---|
ValueError
|
column is missing, riverId repeats, nextRiverId is not a riverId, river drains to itself |
assign_blocks(upstream_counts, downstream_indices, threads)
Divide a region in DFS order into sub-watershed blocks, pack the blocks into threads jobs that route
concurrently, and leave every other river to the main stem, the last job, which routes after them.
A loose cap leaves a short main stem but packs unevenly, a tight one packs evenly but leaves more rivers to the main stem, and the trade turns over at a different cap per network. So each cap of _BLOCK_CAP_FRACTIONS is tried, and the one whose busiest job and main stem together route the fewest rivers is kept. Blocks are kept only when they beat routing every river in one job, which they never do on one thread.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
upstream_counts
|
IntArray
|
(n,) number of rivers upstream of each river, the upstreamCount column |
required |
downstream_indices
|
Int32Array
|
(n,) index of each river's downstream river, -1 at a basin outlet |
required |
threads
|
int
|
number of jobs to pack the blocks into |
required |
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
tuple[JobBlocks, ...]
|
the (block_starts, block_stops) of each job, its blocks in index order, each the rows from a start up to a stop. The last is the main stem, whose blocks are the runs of rows between the sub-watershed blocks. A block's outlet is its last row, the one river of the block that drains out of it. |
analyze_partitioning(network, threads=4)
Static analysis of dividing a network into blocks for multithreaded routing (see assign_blocks).
Reports the block count, how evenly they pack into threads jobs, how much of the network falls into
the sequential main stem, and the speedup that implies. The speedup is an upper bound from work alone: it
assumes perfect scaling and ignores memory bandwidth, which this kernel is largely bound by. Also prints a
human-readable report.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
the network to divide |
required |
threads
|
int
|
thread count the blocks are sized for |
4
|
Returns:
| Name | Type | Description |
|---|---|---|
dict |
dict
|
the summary statistics |
subset_network_to_river(target_river, network_file, out_network_file, weights_file=None, out_weights_file=None)
Subset a network file, and optionally its grid weights, to the target river and every river upstream of it.
In DFS order a river's upstream watershed is the run of rows ending at it, so the subset is the rows whose riverIndex lies from the target's riverIndex minus its upstreamCount to its riverIndex. The target river becomes the outlet (nextRiverId set to -1) of the subset. Every river keeps its riverIndex and upstreamCount, which describe the subset as well as the whole network. With weight table paths the weight table is also filtered to the rivers of the subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target_river
|
int
|
riverId of the river to subset to (becomes the outlet) |
required |
network_file
|
PathInput
|
path to the full network file parquet |
required |
out_network_file
|
PathInput
|
path to write the subset network file parquet |
required |
weights_file
|
PathInput | None
|
path to the full grid weights netCDF file (optional) |
None
|
out_weights_file
|
PathInput | None
|
path to write the subset grid weights netCDF file, given exactly when |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
if the target river is not in the network file, or only one of the weights files is given |