intel/nmt_marian_framework_demo repository overview

⁠Intel® Neural Machine Translation Inference Container for Marian* Framework

The docker container comprises optimally configured environment for the open-source Neural Machine Translation (NMT) Marian* framework.

⁠Package content

The docker* container includes Clear Linux* based environment with the following components:

• Marian* framework binaries & scripts: /nmt/marian
• exemplary input data set: /nmt/data
• pre-trained model: /nmt/model The license files are located in the respective components’ directories. The additional docker container layer contains preinstalled numactl and parallel utilities distributed via Clear Linux* bundle repository (please refer to https://cdn.download.clearlinux.org/releases⁠ for more details).

⁠Hardware requirements

Second Generation Intel® Xeon® Scalable Processors and above.

⁠Documentation and Sources

Marian project webpage: https://marian-nmt.github.io/⁠ WMT’21 submission and result details: https://neural.mt/speed/2021/⁠

⁠License

Marian NMT Framework & helper scripts are published under MIT license. The pretrained models included are published under CC-BY-4.0 license with attribution described in /nmt/model/license.txt

⁠Disclaimer

This package is provided only as a research and benchmarking tool. It is not intended for use in commercial or production environments. The package should not be used for actual translation services as it relies on a model trained with usage of input datasets enlisted on EMNLP Conference website (http://statmt.org/wmt21/translation-task.html⁠, “TRAINING DATA” section) that are known to have potentially significant biases that may adversely affect output. Please review the limitations and bias descriptions for the respective datasets to understand how these limitations and biases may affect the results generated by this model. In particular, the model has the propensity to retain and magnify biases it inherited from any part of its training. This is concerning, since model bias could harm people in the relevant groups in different ways by entrenching existing stereotypes and producing demeaning portrayals amongst other potential harms.