Runtime

The runtime API is available inside a function running on a compute node. Calling instance_info() on the dispatching process raises an error.

Node information

sky.instance_info() returns an Info value. It contains the node id, compute id, rank, peer addresses, local worker slot, and worker concurrency.

import skyward as sky

@sky.function
def report():
    info = sky.instance_info()
    return {
        "rank": info.rank,
        "nodes": info.nodes,
        "worker": info.worker,
        "total_workers": info.total_workers,
        "head": info.head_addr,
    }

The derived properties are:

Property	Meaning
nodes	Number of peer nodes
total_workers	nodes * workers_per_node
global_worker_index	Position of this worker across all node slots
host	Address of this node in peers
head / head_addr	Address of rank zero
head_port	Worker rendezvous port on rank zero
job_id	Compute id
is_head	Whether rank == 0

sky.is_head() is the shorthand for sky.instance_info().is_head.

Sharding

sky.shard() gives the current node its contiguous slice of one or more sequences. All nodes receive the full input and calculate the same rank split locally. With shuffle=True, the same deterministic permutation is applied to each input; seed controls the permutation. drop_last=True truncates the input so every node receives the same number of elements.

@sky.function
def train_epoch(features, labels):
    features, labels = sky.shard(features, labels, shuffle=True, seed=7, drop_last=True)
    return len(features)

Pass node and total_nodes explicitly when testing a shard outside a node. Otherwise they are read from Info.

Output and callbacks

stdout, stderr, and silent control which task output is sent back to the client. redirect_output(callback) temporarily sends both streams to a callback and yields callback writers.

skyward.Info dataclass

The node, as the code running on it sees it.

Attributes:

Name	Type	Description
node	str	The node's id, the same one the daemon and the store use for it.
compute	str	The compute it belongs to.
rank	int	Its position among the compute's nodes, from zero. Stable for as long as the node lives — it is the rank a broadcast froze when it was admitted.
peers	tuple[str, ...]	Every node's address, rank-ordered, this one included. The addresses peers reach each other on, not the ones the daemon dials: the tunnels are the daemon's problem and exist nowhere out here.
worker	int	The task's index among the worker's concurrent slots. Zero in the worker/thread process, distinct per subprocess under executor='process'.
workers_per_node	int	How many concurrent slots the worker runs, the node's concurrency. One unless the node was told to run several tasks at once.

node instance-attribute

compute instance-attribute

rank instance-attribute

peers instance-attribute

worker = 0 class-attribute instance-attribute

workers_per_node = 1 class-attribute instance-attribute

nodes property

total_workers property

Slots across the whole compute, every node's workers_per_node summed.

Uniform here — every node of a compute runs the same concurrency — so it is the node count times the per-node count.

global_worker_index property

This slot's position among all of the compute's slots, from zero.

The rank's block of workers_per_node offset by the local slot, so the slots of rank zero come first, then rank one's, and so on without a gap.

host property

head property

Rank zero's address.

A compute has no head. This is one, by convention, for the libraries that insist on being told where the rendezvous is.

head_addr property

Rank zero's address, under the name the training libraries expect.

head_port property

The port rank zero answers on, the same one every node's worker listens on.

job_id property

The compute's id, under the name the libraries that want a job id expect.

is_head property

__init__(node, compute, rank, peers, worker=0, workers_per_node=1)

skyward.instance_info()

skyward.shard(*data, shuffle=False, seed=None, drop_last=False, node=None, total_nodes=None)

shard(
    data: Sequence[T],
    /,
    *,
    shuffle: bool = False,
    seed: int | None = None,
    drop_last: bool = False,
    node: int | None = None,
    total_nodes: int | None = None,
) -> Sequence[T]

shard(
    data1: Sequence[T1],
    data2: Sequence[T2],
    /,
    *,
    shuffle: bool = False,
    seed: int | None = None,
    drop_last: bool = False,
    node: int | None = None,
    total_nodes: int | None = None,
) -> tuple[Sequence[T1], Sequence[T2]]

shard(
    data1: Sequence[T1],
    data2: Sequence[T2],
    data3: Sequence[T3],
    /,
    *,
    shuffle: bool = False,
    seed: int | None = None,
    drop_last: bool = False,
    node: int | None = None,
    total_nodes: int | None = None,
) -> tuple[Sequence[T1], Sequence[T2], Sequence[T3]]

This node's slice of the data, without the data ever being split anywhere else.

Every node is handed the whole sequence and keeps the part that is its own, which is the cheap way round when the data is already on the machines — a dataset mounted from S3, a file baked into the image — and the only way round that does not put the daemon in the path of every byte.

The split is by rank, contiguous, and identical on every node: they agree because they each compute the same thing, not because anybody told them. Passing several sequences shards each the same way — the same rank split and, under shuffle, the same permutation — so row i of one lines up with row i of the next.

Parameters:

Name	Type	Description	Default
*data	Sequence	One or more sequences. Anything with a length and a slice: a list stays a list, a tuple a tuple, a numpy array or torch tensor its own type — the contiguous slice and the fancy index both preserve it.	()
shuffle	bool	Permute before splitting. Deterministic given seed, which is what makes the nodes' shards disjoint rather than merely random. Equal-length sequences get the same permutation, so their shards stay aligned.	False
seed	int | None	The permutation. None seeds from the compute, so that every node of one compute shuffles the same way and two computes do not.	None
drop_last	bool	Truncate to a multiple of the node count, so every node gets the same number of elements. What a training step usually wants, and what an unbalanced last batch usually breaks.	False
node	int | None	Override this node's rank instead of reading it off the node. For testing, or for sharding against a topology the node is not itself part of.	None
total_nodes	int | None	Override the node count. Given together with node, the split needs no node to run on at all.	None

Returns:

Type	Description
Sequence | tuple[Sequence, ...]	This rank's elements — one sequence for one argument, a tuple of them for several. Possibly empty: eight nodes and three items is three nodes with work and five without, not an error.

skyward.is_head()

skyward.stdout(fn=None, *, only=None)

stdout(fn: Callable[P, T]) -> Callable[P, T]

stdout(
    *, only: OutputSpec
) -> Callable[[Callable[P, T]], Callable[P, T]]

Forward stdout and drop stderr, optionally from only some of the nodes.

only is a rank, a sequence of ranks, "head" for rank zero, or a predicate on the node's own :class:Info. only="head" is what turns a broadcast onto sixty-four nodes back into something a human can read.

skyward.stderr(fn=None, *, only=None)

stderr(fn: Callable[P, T]) -> Callable[P, T]

stderr(
    *, only: OutputSpec
) -> Callable[[Callable[P, T]], Callable[P, T]]

Forward stderr and drop stdout, optionally from only some of the nodes.

skyward.silent(fn)

Say nothing. The result still comes back; the printing does not.

skyward.CallbackWriter

Bases: TextIOBase

A writable stream that hands every write to a callback and keeps nothing.

__init__(callback)

write(s)

writable()

isatty()

skyward.redirect_output(callback)

Send the running function's stdout and stderr to callback for the block.

Both streams are redirected into a :class:CallbackWriter, so print and anything writing to sys.stderr reach callback a write at a time. Runs on the node, inside the task; the redirection ends with the block.

Parameters:

Name	Type	Description	Default
callback	Callable[[str], None]	Called with each string written to either stream.	required

Yields:

Type	Description
tuple[CallbackWriter, CallbackWriter]	The (stdout, stderr) writers now standing in for the two streams.