Internals
ska_telmodel._common
- class ska_telmodel._common.TMSchema(schema: Optional[Any] = None, error=None, ignore_extra_keys: bool = False, name: Optional[str] = None, description: Optional[str] = None, as_reference: bool = False, version: Optional[str] = None, strict: bool = False)[source]
Wrapper on top of schema.Schema for incremental schema build-up.
- ska_telmodel._common.get_unique_id_schema(strict: bool, type_re: str = '[a-z0-9]+') schema.Schema [source]
Return schema for unique identifier.
- Parameters
type_re – Restricts ID type(s) to accept.
- ska_telmodel._common.interface_uri(prefix: str, *versions: int) str [source]
Make an URI from the given prefix and versions
- Parameters
prefix – Schema URI prefix. Must end in ‘/’
versions – Components of the version
ska_telmodel.channel_map
Tools for working with JSON compressed channel maps.
The SKA is meant to have a large number of channels, which means that any type of per-channel configuration might become very cumbersome to transfer and reason about. To prevent such issues we are using a simple run-length encoding to “compress” the representation. The idea is that if we write:
[ [0,0], [200,1], [400, 3] ]
We essentially mean the dictionary:
{ 0: 0, 1: 0, ..., 199:0, 200:1, ..., 399:1, 400: 3, ...}
Furthermore runs of numbers are supported, by adding an increment:
[ [0,0,1], [200,1] ]
Means:
{ 0: 0, 1: 1, 2:2, ..., 199:100, 200:1, ...}
- ska_telmodel.channel_map.channel_map_at(channel_map: List[list], channel: int, make_entry: bool = False) Any [source]
Query a value from a channel map
- Parameters
channel_map – Queried map
channel – Channel ID to query
make_entry – Return an channel map entry (including increment) instead of just the value
- Returns
Value from map
- ska_telmodel.channel_map.shift_channel_map(channel_map: List[list], channel_shift: int) List[list] [source]
Shift a channel map by some channel distance
- Parameters
channel_map – Channel map to use
channel_shift – Shift to apply
- ska_telmodel.channel_map.split_channel_map(channel_map: List[list], first_channel: int, channel_group_steps: int, rebase_groups: Optional[int] = None, minimum_groups: int = 0) List[List[list]] [source]
Split a channel map using a constant channel step length
- Parameters
channel_map – Channel map to split. Each entry is expected to have the start channel in the first field, and mapped data in the remaining entries
first_channel – First channel to appear in the map
channel_group_steps – Chunks to split the channel map into
rebase_groups – Start every group at given channel index (None: left as-is)
minimum_groups – Minimum number of groups to return
- Returns
List of channel maps
- ska_telmodel.channel_map.split_channel_map_at(channel_map: List[list], channel_groups: List[int], rebase_groups: Optional[int] = None) List[List[list]] [source]
Split a channel map at certain points
- Parameters
channel_map – Channel map to split. Each entry is expected to have the start channel in the first field, and mapped data in the remaining entries
channel_groups – Boundaries between channel groups. The n-th returned channel map will cover channels channel_groups[n]..channel_groups[n+1]-1. Needs to have at least two entries.
rebase_groups – Start every group at given channel index (None: left as-is)
- Returns
List of channel maps
ska_telmodel.data
- class ska_telmodel.data.backend.CARBackend(uri: str, *args, **kwargs)[source]
Represents data in (a mirror of) the SKA central artefact repository. Permissible URI formats:
car:[project name]?[branch]#[directory] car://[gitlab server]/[project name]?[branch]#[directory]
So for instance:
car:ska-telmodel?master car://gitlab.com/ska-telescope/ska-telmodel?master#tmdata
The source of truth might still be Gitlab, yet this backend will only work with artefacts that have been uploaded to the CAR. The short form URI will be expanded into the long form automatically.
- class ska_telmodel.data.backend.FilesystemBackend(uri: str, update: bool = False)[source]
Retrives data from a locally accessible file system. URI format:
file://[absolute path]
Note that changes to the file system are outside of our control. Consistency must be ensured externally.
- classmethod backend_name() str [source]
Returns the name of the backend.
Will be used for the scheme in URIs to identify the backend type of a telescope model data source.
- copy(key: str, dest: str)[source]
Write key contents to a file.
Raises KeyError if the key does not exist
- Parameters
key – Key to query
dest – Path of destination file
- exists(key: str) bool [source]
Check whether a given key exists.
- Parameters
key – Key to query
- Returns
True if key exists
- get(key: str) bytes [source]
Get the data stored with the given key
- Parameters
key – Key to query
- Returns
Bytes stored at key
- get_uri(pinned: bool) str [source]
Returns URI for this telescope model data backend
- Parameters
pinned – Attempt to return an URI that will continue to refer to this specific version of telescope model data
- Returns
URI identifying data source
- list_keys(key_prefix: str = '') collections.abc.Iterable[str] [source]
List children keys
Yields all keys with prefix “{key_prefix}/” in ascending order
- Parameters
key_prefix – Path to query
- class ska_telmodel.data.backend.GitlabBackend(uri: str, update: bool = False, gl: gitlab.Gitlab = None, try_nexus: bool = True, nexus_url: str = None, env=None)[source]
Represents data in a GitLab repository. URI format:
gitlab://[gitlab server]/[project name]?[branch]#[directory]
So for instance:
gitlab://gitlab.com/ska-telescope/ska-telmodel?master#tmdata
Would refer to data contained in the
ska-telmodel
repository itself.Repositories accessed in this way should make sure to activate the
tmdata
standard continuous integration stages (see https://gitlab.com/ska-telescope/templates-repository ) to ensure that telescope model data is cached in the SKAO central artefact repository. Once that has been done, this library will never actually query GitLab directly.Furthermore, this backend will cache all loaded data locally, including resolved Gitlab references (like
master
in the example above). This especially means that once instantiated, the version of data will be “pinned” even between different instances (and processes). Use theupdate
parameter toska_telmodel.data.TMData
orGitlabBackend
respectively to refresh the local cache.- classmethod backend_name() str [source]
Returns the name of the backend.
Will be used for the scheme in URIs to identify the backend type of a telescope model data source.
- copy(key: str, dest: str)[source]
Write key contents to a file.
Raises KeyError if the key does not exist
- Parameters
key – Key to query
dest – Path of destination file
- exists(key: str) bytes [source]
Check whether a given key exists.
- Parameters
key – Key to query
- Returns
True if key exists
- get(key: str) bytes [source]
Get the data stored with the given key
- Parameters
key – Key to query
- Returns
Data stored at key, or None if it doesn’t exist
- get_uri(pinned: bool) str [source]
Returns URI for this telescope model data backend
- Parameters
pinned – Attempt to return an URI that will continue to refer to this specific version of telescope model data
- Returns
URI identifying data source
- list_keys(key_prefix: str = '') collections.abc.Iterable[str] [source]
List children keys
Yields all keys with prefix “{key_prefix}/” in ascending order. Exception is if the path is empty, in which case all available keys are listed.
- Parameters
key_prefix – Path to query
- class ska_telmodel.data.backend.MemoryBackend(uri: str, update: bool = False)[source]
Represents in-memory data. URIs should look as follows:
mem://?[key1]=[value1]&[key2]=[value2]
This will directly set the given telescope model data keys to the given values. Useful for testing, and overriding single values in telescope model data.
- classmethod backend_name() str [source]
Returns the name of the backend.
Will be used for the scheme in URIs to identify the backend type of a telescope model data source.
- get(key: str) bytes [source]
Get the data stored with the given key
- Parameters
key – Key to query
- Returns
Bytes stored at key
- get_uri(pinned: bool) str [source]
Returns URI for this telescope model data backend
- Parameters
pinned – Attempt to return an URI that will continue to refer to this specific version of telescope model data
- Returns
URI identifying data source
- list_keys(key_prefix: str = '') collections.abc.Iterable[str] [source]
List children keys
Yields all keys with prefix “{key_prefix}/” in ascending order
- Parameters
key_prefix – Path to query
- class ska_telmodel.data.backend.TMDataBackend(uri: str, update: bool = False)[source]
Base class for telescope model data backends
Sub-classes should override
backend_name()
, then utilisetelmodel_backend()
to register the telescope model data backend. A minimal implementation should furthermore providelist_keys()
andget()
.- abstract classmethod backend_name() str [source]
Returns the name of the backend.
Will be used for the scheme in URIs to identify the backend type of a telescope model data source.
- copy(key: str, dest: str)[source]
Write key contents to a file.
Raises KeyError if the key does not exist
- Parameters
key – Key to query
dest – Path of destination file
- exists(key: str) bool [source]
Check whether a given key exists.
- Parameters
key – Key to query
- Returns
True if key exists
- abstract get(key: str) bytes [source]
Get the data stored with the given key
- Parameters
key – Key to query
- Returns
Data stored at key, or None if it doesn’t exist
- get_uri(pinned: bool) str [source]
Returns URI for this telescope model data backend
- Parameters
pinned – Attempt to return an URI that will continue to refer to this specific version of telescope model data
- Returns
URI identifying data source
- abstract list_keys(key_prefix: str = '') collections.abc.Iterable[str] [source]
List children keys
Yields all keys with prefix “{key_prefix}/” in ascending order. Exception is if the path is empty, in which case all available keys are listed.
- Parameters
key_prefix – Path to query
- open(key: str) IO[bytes] [source]
Access data at given key as a file-like object
Raises KeyError if the key does not exist
- Parameters
key – Key to query
ska_telmodel.csp
- ska_telmodel.csp.config.add_midcbf_visibility_receive_addresses(csp_config: dict, scan_receive_addrs: dict, csp_interface_version: str, sdp_interface_version: str) dict [source]
Add SDP visibility receive addresses into mid-cbf configuration
- Parameters
csp_config – CSP input configuration
scan_receive_addrs – SDP receive addresses for scan
csp_interface_version – CSP interface version to assume
sdp_interface_version – SDP interface version to assume
- Returns
New CSP configuration
- ska_telmodel.csp.config.add_pss_receive_addresses(csp_config: dict, scan_receive_addrs: dict, csp_interface_version: str, sdp_interface_version: str) dict [source]
Add SDP visibility receive addresses into pulsar search configuration
- Parameters
csp_config – CSP input configuration
scan_receive_addrs – SDP receive addresses for scan
csp_interface_version – CSP interface version to assume
sdp_interface_version – SDP interface version to assume
- Returns
New CSP configuration
- ska_telmodel.csp.config.add_pst_receive_addresses(csp_config: dict, scan_receive_addrs: dict, csp_interface_version: str, sdp_interface_version: str) dict [source]
Add SDP visibility receive addresses into pulsar timing configuration
- Parameters
scan_trype – Scan type executed
csp_config – CSP input configuration
sdp_receive_addrs – SDP receive addresses for scan
- Returns
New CSP configuration
- ska_telmodel.csp.config.add_receive_addresses(scan_type: str, csp_config: dict, scan_receive_addrs: dict, csp_interface_version: str, sdp_interface_version: str) dict [source]
Add SDP visibility receive addresses into CSP configuration
- Parameters
csp_config – CSP input configuration
scan_receive_addrs – SDP receive addresses for scan
csp_interface_version – CSP interface version to assume
sdp_interface_version – SDP interface version to assume
- Returns
New CSP configuration
- ska_telmodel.csp.config.get_fsp_channel_offset(csp_config_in: dict) int [source]
Determines first channel ID within an FSP
- ska_telmodel.csp.config.get_fsp_output_channel_offset(fsp_config: dict, fsp_id: str, fsp_ch_offset: str) int [source]
Determines the FSP channel offset. Either read from the dictionary or reconstructed.
- Parameters
fsp_config – FSP configuration structure
fsp_id – Position of FSP in configuration
fsp_ch_offset – Name of FSP channel offset field
- ska_telmodel.csp.examples.get_csp_assignresources_example(version: str) dict [source]
Generate example of CSP assignresources argument
- Parameters
version – Version URI of configuration format
- ska_telmodel.csp.examples.get_csp_config_example(version: str, scan: Optional[str] = None) dict [source]
Generate examples for CSP configuration strings
- Parameters
version – Version URI of configuration format
scan – Includes SDP receive addresses for a scan? None means that this is “template” configuration as passed to TMC. Valid parameters: cal_a, science_a
- ska_telmodel.csp.examples.get_csp_delaymodel_example(version: str) dict [source]
Generate example of CSP delay model argument
- Parameters
version – Version URI of configuration format
- ska_telmodel.csp.examples.get_csp_endscan_example(version: str) dict [source]
Generate example of CSP endscan argument
- Parameters
version – Version URI of configuration format
- ska_telmodel.csp.examples.get_csp_releaseresources_example(version: str) dict [source]
Generate example of CSP releaseresources argument
- Parameters
version – Version URI of configuration format
- ska_telmodel.csp.examples.get_csp_scan_example(version: str) dict [source]
Generate example of CSP scan argument
- Parameters
version – Version URI of configuration format
csp.validators module defines constants and functions for validating CSP fields in schemas.
- ska_telmodel.csp.validators.validate_integration_factor(integration_factor: int) bool [source]
Checks if the integration_factor is valid.
- Parameters
integration_factor – Integration time for correlation products
- Returns
True if integration_factor is valid
- ska_telmodel.csp.version.check_csp_interface_version(version: str, allowed_prefixes: Union[str, List[str]] = ['https://schema.skao.int/ska-csp-assignresources/', 'https://schema.skao.int/ska-csp-configure/', 'https://schema.skao.int/ska-csp-scan/', 'https://schema.skao.int/ska-csp-endscan/', 'https://schema.skao.int/ska-csp-releaseresources/', 'https://schema.skao.int/ska-csp-delaymodel/']) str [source]
Check CSP interface version.
Checks that the interface URI has one of the allowed prefixes. If it does, the version number is returned. If not, a ValueError exception is raised.
- Parameters
version – CSP interface URI
allowed_prefixes – allowed URI prefix(es)
- Returns
version number
- ska_telmodel.csp.version.csp_config_versions(min_ver=None, max_ver=None)[source]
Returns a list of CSP configuration interface version URIs
- Parameters
min_ver – Tuple of minimum version to return
max_ver – Tuple of maximum version to return
- ska_telmodel.csp.version.normalize_csp_config_version(csp_interface_version: Union[int, str], csp_config: Optional[dict] = None)[source]
Provides a standard interface version for configure
- Parameters
csp_interface_version – External guess at the interface version
csp_config – Example configuration to derive version from
- Returns
Canonical URI of interface version
ska_telmodel.sdp
Define schemas for SDP commands.
Miscellaneous schemas that probably should be moved somewhere else.
- ska_telmodel.sdp.common.ALL_RECEPTORS = Or(Regex('^C([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-1][0-9]|22[0-4])$'), Regex('^[ENS]([1-9]|1[0-6])-[1-6]$'), Regex('^FS([1-9]|[1-9][0-9]|[1-4][0-9][0-9]|50[0-9]|51[0-2])(\\.\\S+)?$'), Regex('^SKA((?!000)0[0-9][0-9]|1[0-2][0-9]|13[0-3])$'), Regex('^MKT0([0-5][0-9]|6[0-3])$'))
All receptors.
- ska_telmodel.sdp.common.LOW_CORE = Regex('^C([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-1][0-9]|22[0-4])$')
LOW core receptors, 1-224
- ska_telmodel.sdp.common.LOW_DIRS = Regex('^[ENS]([1-9]|1[0-6])-[1-6]$')
LOW east/north/south receptors.
- ska_telmodel.sdp.common.LOW_FS = Regex('^FS([1-9]|[1-9][0-9]|[1-4][0-9][0-9]|50[0-9]|51[0-2])(\\.\\S+)?$')
LOW FS 1-512, plus optional substations.
- ska_telmodel.sdp.common.MID_MKT = Regex('^MKT0([0-5][0-9]|6[0-3])$')
MID Meerkat, 000-063.
- ska_telmodel.sdp.common.MID_SKA = Regex('^SKA((?!000)0[0-9][0-9]|1[0-2][0-9]|13[0-3])$')
MID SKA, 001-133.
- ska_telmodel.sdp.common.get_beam_function_pattern(strict: bool)[source]
Get pattern for SDP beam functions
As used for SDP configuration - i.e. basically a kind of data that the SKA SDP needs to receive.
- Returns
A string pattern suitable for use in schemas
- ska_telmodel.sdp.common.get_receptor_schema(strict: bool) schema.Schema [source]
Return schema for receptors.
- Parameters
strict – check names if set
- Returns
schema
- ska_telmodel.sdp.examples.get_sdp_assignres_example(version: Union[int, str]) dict [source]
Generate example of SDP assign resources argument.
- Parameters
version – SDP assign resources version
- Returns
Example dictionary
- ska_telmodel.sdp.examples.get_sdp_configure_example(version: Union[int, str], scan_type: str = 'science') dict [source]
Generate example of SDP configure argument.
- Parameters
version – SDP configure version
scan_type – Scan type to configure. “new_calibration” declares a new scan in-line.
- Returns
Example dictionary
- ska_telmodel.sdp.examples.get_sdp_recvaddrs_example(version: Union[int, str]) dict [source]
Generate example of SDP receive addresses map.
- Parameters
version – SDP receive addresses version
- Returns
Example dictionary
- ska_telmodel.sdp.examples.get_sdp_releaseres_example(version: Union[int, str]) dict [source]
Generate example of SDP release resources argument.
- Parameters
version – SDP release resources version
- Returns
Example dictionary
- ska_telmodel.sdp.examples.get_sdp_scan_example(version: Union[int, str], scan_id: int = 1) dict [source]
Generate example of SDP scan argument.
- Parameters
version – SDP scan version
scan_id – Scan ID to start
- Returns
Example dictionary
Define processing blocks schemas.
Defines receive addresses schema.
Used for checking SDP strings for conformance.
- ska_telmodel.sdp.version.CALL_SIG
Call signature for schemas.
- ska_telmodel.sdp.version.PREFIXES_TYPE
The type af allowed prefixes.
- class ska_telmodel.sdp.version.SchemaFactory(prefix: Optional[str] = None, allowed_prefixes: Optional[Union[str, Sequence[str]]] = None)[source]
Get the right schema for a type based on its version.
- get_schema(version: ska_telmodel.sdp.version.SdpVersion, strict: bool) schema.Schema [source]
Get the schema for this version.
If strict, an exact match is required. Otherwise, the last minor version matching the major version is accepted. It is assumed that a version is of the form version.subversion.
- Parameters
version – SDP version object
strict – whether strict or not
- Returns
the matching schema
- class ska_telmodel.sdp.version.SdpVersion(version: Union[int, str], prefix: Optional[str] = None, allowed_prefixes: Optional[Union[str, Sequence[str]]] = None)[source]
Wrapper around the normalise/check functions and stores the results.
- Parameters
version – SDP interface version
prefix – schema prefix
allowed_prefixes – allowed URI prefix(es)
- Returns
version object
- ska_telmodel.sdp.version.check_sdp_interface_version(version: str, allowed_prefixes: Optional[Union[str, Sequence[str]]] = None) str [source]
Check SDP interface version.
Checks that the interface URI has one of the allowed prefixes. If it does, the version number is returned. If not, a ValueError exception is raised.
- Parameters
version – SDP interface URI
allowed_prefixes – allowed URI prefix(es)
- Returns
version number
- ska_telmodel.sdp.version.normalise_sdp_interface_version(version: Union[int, str], prefix: str) str [source]
Normalise SDP interface version.
Converts deprecated integer version number into a schema URI, where the prefix specifies which schema to use. If the version is a string, it is assumed to be a schema URI and it is returned unchanged.
- Parameters
version – SDP interface version
prefix – schema prefix
- Returns
SDP interface URI