Rucio - Scientific Data Management (Daemon Container, For Public Use)
1M+
Rucio is a software framework that provides functionality to organize, manage, and access large volumes of scientific data using customisable policies. The data can be spread across globally distributed locations and across heterogeneous data centers, uniting different storage and network technologies as a single federated entity. Rucio offers advanced features such as distributed data recovery or adaptive replication, and is highly scalable, modular, and extensible. Rucio has been originally developed to meet the requirements of the high-energy physics experiment ATLAS, and is continuously extended to support LHC experiments and other diverse scientific communities.
General information and latest documentation about Rucio can be found at readthedocs or on our webpage.
For information on how to contribute to Rucio, please refer and follow our CONTRIBUTING guidelines.
This image provides the Rucio daemons. Each daemon has to be run in a separate container. It supports MySQL, PostgreSQL, Oracle and SQLite as database backends.
This image expects that there is an already initialised Rucio DB. To start a simple judge-cleaner daemon using a database on mysql.db without an additional parameters just run this:
docker run \
--name=rucio-judge-cleaner \
-e RUCIO_CFG_DATABASE_DEFAULT="mysql://rucio:[email protected]/rucio" \
-e RUCIO_DAEMON=judge-cleaner \
rucio/rucio-daemons
The RUCIO_DAEMON environment variable gives the name of the rucio daemon.
Rucio can be configured fully using environment variables like RUCIO_CFG_DATABASE_DEFAULT. If you want to instead use a complete rucio.cfg it can also be mounted. This will then ignore the RUCIO_CFG environment variables:
The rucio.cfg is used to configure the database backend and the daemons:
docker run \
--name=rucio-judge-cleaner \
-v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
-e RUCIO_DAEMON=judge-cleaner \
rucio/rucio-daemons
By default the daemon logs are written to stdout and stderr if you want to write to a file you can use RUCIO_ENABLE_LOGS like this:
docker run \
--name=rucio-judge-cleaner \
-v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
-v /tmp/logs:/var/log/rucio \
-e RUCIO_DAEMON=judge-cleaner \
-e RUCIO_ENABLE_LOGS=True \
rucio/rucio-daemons
As shown in the examples above the rucio-daemon image can be configured using environment variables that are passed with docker run. Below is a list of all available variables and their behaviour:
RUCIO_DAEMONThis variable is mandatory and it specifies the name of the daemon, e.g., hermes, kronos, judge-evaluator, etc.
RUCIO_DAEMON_ARGSAny additional command line parameter can be specified here, e.g., --run-once. This field is optional.
RUCIO_ENABLE_LOGSBy default the log output of the daemon is written to stdout and stderr. If you set this variable to True the output will be written to access_log and error_log under /var/log/rucio.
RUCIO_PRINT_CFGThis variable can be used to print the content of autogenerated rucio.cfg to the logs. It will be printed once at the daemon startup.
RUCIO_CFG configuration parameters:Environment variables can be used to set values for the auto-generated rucio.cfg. The names are derived from the actual names in the configuration file prefixed by RUCIO_CFG, e.g., the default value in the database section becomes RUCIO_CFG_DATABASE_DEFAULT.
The available environment variables are:
RUCIO_CFG_ACCOUNTS_SPECIAL_ACCOUNTSRUCIO_CFG_AUDITOR_CACHERUCIO_CFG_AUDITOR_RESULTSRUCIO_CFG_AUTOMATIX_DATASET_LIFETIMERUCIO_CFG_AUTOMATIX_SET_METADATARUCIO_CFG_AUTOMATIX_SITESRUCIO_CFG_AUTOMATIX_SLEEP_TIMERUCIO_CFG_CLIENT_ACCOUNTRUCIO_CFG_CLIENT_AUTH_HOSTRUCIO_CFG_CLIENT_AUTH_TYPERUCIO_CFG_CLIENT_CA_CERTRUCIO_CFG_CLIENT_CLIENT_CERTRUCIO_CFG_CLIENT_CLIENT_KEYRUCIO_CFG_CLIENT_CLIENT_VORUCIO_CFG_CLIENT_CLIENT_X509_PROXYRUCIO_CFG_CLIENT_PASSWORDRUCIO_CFG_CLIENT_REQUEST_RETRIESRUCIO_CFG_CLIENT_RUCIO_HOSTRUCIO_CFG_CLIENT_USERNAMERUCIO_CFG_COMMON_LOGDIRRUCIO_CFG_COMMON_LOGLEVELRUCIO_CFG_COMMON_MAILTEMPLATEDIRRUCIO_CFG_COMMON_MULTI_VORUCIO_CFG_CONVEYOR_ALLOW_USER_OIDC_TOKENSRUCIO_CFG_CONVEYOR_BRING_ONLINERUCIO_CFG_CONVEYOR_CACERTRUCIO_CFG_CONVEYOR_CACHE_TIMERUCIO_CFG_CONVEYOR_FTSHOSTSRUCIO_CFG_CONVEYOR_FTSMONHOSTSRUCIO_CFG_CONVEYOR_POLL_TIMEOUTRUCIO_CFG_CONVEYOR_QUEUE_MODERUCIO_CFG_CONVEYOR_REQUEST_OIDC_AUDIENCERUCIO_CFG_CONVEYOR_REQUEST_OIDC_SCOPERUCIO_CFG_CONVEYOR_SCHEMERUCIO_CFG_CONVEYOR_SUBMIT_TIMEOUTRUCIO_CFG_CONVEYOR_TRANSFERTOOLRUCIO_CFG_CONVEYOR_USE_DETERMINISTIC_IDRUCIO_CFG_CONVEYOR_USER_ACTIVITIESRUCIO_CFG_CONVEYOR_USERCERTRUCIO_CFG_CONVEYOR_USER_TRANSFERSRUCIO_CFG_CONVEYOR_USING_MEMCACHERUCIO_CFG_CREDENTIALS_GCSRUCIO_CFG_CREDENTIALS_SIGNATURE_LIFETIMERUCIO_CFG_DATABASE_DEFAULTRUCIO_CFG_DATABASE_ECHORUCIO_CFG_DATABASE_MAX_OVERFLOWRUCIO_CFG_DATABASE_POOL_RECYCLERUCIO_CFG_DATABASE_POOL_RESET_ON_RETURNRUCIO_CFG_DATABASE_POOL_SIZERUCIO_CFG_DATABASE_POOL_TIMEOUTRUCIO_CFG_DATABASE_POWUSERACCOUNTRUCIO_CFG_DATABASE_POWUSERPASSWORDRUCIO_CFG_DATABASE_SCHEMARUCIO_CFG_MESSAGING_CACHE_ACCOUNTRUCIO_CFG_MESSAGING_CACHE_BROKERSRUCIO_CFG_MESSAGING_CACHE_DESTINATIONRUCIO_CFG_MESSAGING_CACHE_PORTRUCIO_CFG_MESSAGING_CACHE_SSL_CERT_FILERUCIO_CFG_MESSAGING_CACHE_SSL_KEY_FILERUCIO_CFG_MESSAGING_CACHE_VONAMERUCIO_CFG_MESSAGING_FTS3_BROKERSRUCIO_CFG_MESSAGING_FTS3_DESTINATIONRUCIO_CFG_MESSAGING_FTS3_NONSSL_PORTRUCIO_CFG_MESSAGING_FTS3_PASSWORDRUCIO_CFG_MESSAGING_FTS3_PORTRUCIO_CFG_MESSAGING_FTS3_SSL_CERT_FILERUCIO_CFG_MESSAGING_FTS3_SSL_KEY_FILERUCIO_CFG_MESSAGING_FTS3_USERNAMERUCIO_CFG_MESSAGING_FTS3_USE_SSLRUCIO_CFG_MESSAGING_FTS3_VONAMERUCIO_CFG_MESSAGING_HERMES_BROKERSRUCIO_CFG_MESSAGING_HERMES_BROKER_VIRTUAL_HOSTRUCIO_CFG_MESSAGING_HERMES_DESTINATIONRUCIO_CFG_MESSAGING_HERMES_EMAIL_FROMRUCIO_CFG_MESSAGING_HERMES_EMAIL_TESTRUCIO_CFG_MESSAGING_HERMES_NONSSL_PORTRUCIO_CFG_MESSAGING_HERMES_PASSWORDRUCIO_CFG_MESSAGING_HERMES_PORTRUCIO_CFG_MESSAGING_HERMES_SSL_CERT_FILERUCIO_CFG_MESSAGING_HERMES_SSL_KEY_FILERUCIO_CFG_MESSAGING_HERMES_USERNAMERUCIO_CFG_MESSAGING_HERMES_USE_SSLRUCIO_CFG_MESSAGING_HERMES_VONAMERUCIO_CFG_MONITOR_CARBON_PORTRUCIO_CFG_MONITOR_CARBON_SERVERRUCIO_CFG_MONITOR_ENABLE_METRICSRUCIO_CFG_MONITOR_USER_SCOPERUCIO_CFG_NAGIOS_FTS_VORUCIO_CFG_NAGIOS_PROXYRUCIO_CFG_NAGIOS_UPDATE_DISTANCESRUCIO_CFG_OIDC_ADMIN_ISSUERRUCIO_CFG_OIDC_EXPECTED_AUDIENCERUCIO_CFG_OIDC_IDPSECRETSRUCIO_CFG_POLICY_LFN2PFN_ALGORITHM_DEFAULTRUCIO_CFG_POLICY_LFN2PFN_MODULERUCIO_CFG_POLICY_PACKAGERUCIO_CFG_POLICY_PERMISSIONRUCIO_CFG_POLICY_SCHEMARUCIO_CFG_POLICY_SUPPORTRUCIO_CFG_POLICY_SUPPORT_RUCIORUCIO_CFG_TRACER_KRONOS_BROKERSRUCIO_CFG_TRACER_KRONOS_CHUNKSIZERUCIO_CFG_TRACER_KRONOS_DATASET_WAITRUCIO_CFG_TRACER_KRONOS_EXCLUDED_USRDNSRUCIO_CFG_TRACER_KRONOS_PASSWORDRUCIO_CFG_TRACER_KRONOS_PORTRUCIO_CFG_TRACER_KRONOS_PREFETCH_SIZERUCIO_CFG_TRACER_KRONOS_QUEUERUCIO_CFG_TRACER_KRONOS_RECONNECT_ATTEMPTSRUCIO_CFG_TRACER_KRONOS_SSL_CERT_FILERUCIO_CFG_TRACER_KRONOS_SSL_KEY_FILERUCIO_CFG_TRACER_KRONOS_SUBSCRIPTION_IDRUCIO_CFG_TRACER_KRONOS_USERNAMERUCIO_CFG_TRACER_KRONOS_USE_SSLRUCIO_CFG_TRANSMOGRIFIER_MAXDIDSIf you are looking for support, please contact our mailing list [email protected] or join us on our slack support channel.
Content type
Image
Digest
sha256:ec16136db…
Size
350.2 MB
Last updated
12 days ago
docker pull rucio/rucio-daemonsPulls:
1,423
Jul 27 to Aug 2