Databricks Runtime 8.4 for ML(不支持)
Databricks於2021年7月發布了這張圖片。
Databricks Runtime 8.4 for Machine Learning為機器學習和數據科學提供了一個現成的環境Databricks Runtime 8.4(不支持).Databricks Runtime ML包含許多流行的機器學習庫,包括TensorFlow、PyTorch和XGBoost。它還支持使用Horovod進行分布式深度學習訓練。
有關更多信息,包括創建Databricks Runtime ML集群的說明,請參見介紹Databricks運行時機器學習.
新功能和改進
Databricks Runtime 8.4 ML是建立在Databricks Runtime 8.4之上的。有關Databricks Runtime 8.4(包括Apache Spark MLlib和SparkR)中的新特性的信息,請參見Databricks Runtime 8.4(不支持)發行說明。
FeatureStoreClient v0.3.2
允許特性和特性表名稱與SQL保留字衝突。
驗證所提供的數據幀是否為PySpark數據幀(
pyspark.sql.dataframe.DataFrame
).
AutoML v1.1.0
Databricks Runtime 8.4 ML附帶的AutoML更新版本包括一些錯誤修複和穩定性改進。
AutoML分類現在也運行試驗LGBMClassifier
AutoML回歸現在也運行試驗LGBMRegressor
Databricks Runtime ML Python環境的主要更改
看到Databricks Runtime 8.4(不支持)參閱Databricks Runtime Python環境的主要更改。有關已安裝的Python包及其版本的完整列表,請參見Python庫.
的用法
以下更改已棄用,並將在Databricks Runtime 9.0中刪除:
在HorovodRunner中,設置
np = 0
,在那裏np
是用於Horovod作業的並行進程數。英特爾數學內核庫(Intel MKL),以及依賴它的下遊包。
的
azure-core
python庫用於核心Azure異常和模塊的
azure-storage-blob
python庫客戶端用於與Azure Storage Blob服務交互的
msr
python庫用於AutoRest swagger生成的
碼頭工人
Docker引擎API的python庫的
querystring-parser
python庫用於在python /Django中解析查詢的
intel-openmp
Python庫,用於創建多線程軟件
係統環境
Databricks Runtime 8.4 ML的係統環境與Databricks Runtime 8.4不同:
DBUtils: Databricks Runtime ML不包含庫實用程序(dbutls . Library).使用
%皮普
而且% conda
而不是命令。看到筆記本範圍的Python庫.對於GPU集群,Databricks Runtime ML包含以下NVIDIA GPU庫:
CUDA 11.0
cuDNN 8.0.4.30
NCCL 2.7.8
TensorRT 7.1.3
庫
以下部分列出了Databricks Runtime 8.4 ML中包含的不同於Databricks Runtime 8.4的庫。
Python庫
Databricks Runtime 8.4 ML使用Conda進行Python包管理,包括許多流行的ML包。
除了以下章節中Conda環境中指定的包外,Databricks Runtime 8.4 ML還包括以下包:
hyperopt 0.2.5.db2
sparkdl 2.1.0.db4
feature_store 0.3.2
automl 1.1.0
CPU集群上的Python庫
名字:databricks-ml渠道:-pytorch-違約依賴關係:-_libgcc_mutex = 0.1 =主要-absl-py = 0.11.0 = pyhd3eb1b0_1它-aiohttp = 3.7.4 = py38h27cfd23_1-asn1crypto = 1.4.0 = py_0-阿斯特= 0.8.1 = py38h06a4308_0-async-timeout = 3.0.1 = py38h06a4308_0-attrs = 20.3.0 = pyhd3eb1b0_0-backcall = 0.2.0 = pyhd3eb1b0_0-bcrypt = 3.2.0 = py38h7b6447c_0-布拉斯特區= 1.0 = mkl-信號燈= 1.4 = py38h06a4308_0-boto3 = 1.16.7 = pyhd3eb1b0_0-botocore = 1.19.7 = pyhd3eb1b0_0-brotlipy = 0.7.0 = py38h27cfd23_1003-bzip2 = 1.0.8 = h7b6447c_0-ca證書= 2021.5.25 = h06a4308_1-cachetools = 4.2.2 = pyhd3eb1b0_0-certifi = 2021.5.30 = py38h06a4308_0-cffi = 1.14.3 = py38h261ae71_2-chardet = 3.0.4 = py38h06a4308_1003-單擊= 7.1.2 = pyhd3eb1b0_0-cloudpickle = 1.6.0 = py_0-configparser = 5.0.1 = py_0-cpuonly = 1.0 = 0-密碼= 3.1.1 = py38h1ba5d50_0-周期計= 0.10.0 = py38_0-cython = 0.29.21 = py38h2531618_0-decorator = 10/24/11 = pyhd3eb1b0_0-蒔蘿= 0.3.2 = py_0-docutils = 0.15.2 = py38h06a4308_1-entrypoints = 0.3 = py38_0-ffmpeg = 4.2.2 = h20bf706_0-瓶= 1.1.2 = pyhd3eb1b0_0-freetype的= 2.10.4 = h5ab3b9f_0-fsspec = 0.8.3 = py_0-未來= 0.18.2 = py38_1-恐嚇= 0.4.0 = py_0-gitdb = 4.0.7 = pyhd3eb1b0_0-gitpython = 3.1.12 = pyhd3eb1b0_1-6.1.2 gmp = = h6c8ec71_1-gnutls = 3.6.15 = he1e5248_0-google-auth = 1.22.1 = py_0-google-auth-oauthlib = 0.4.2 = pyhd3eb1b0_2-google-pasta = 0.2.0 = py_0-gunicorn = 20.0.4 = py38h06a4308_0-hdf5 = 1.10.4 = hb1b8bf9_0-icu = 58.2 = he6710b0_3-idna = 2.10 = pyhd3eb1b0_0-importlib-metadata = 2.0.0 = py_1-intel-openmp = 2019.4 = 243-ipykernel = 5.3.4 = py38h5ca1d4c_0-ipython = 7.19.0 = py38hb070fc8_1-ipython_genutils = 0.2.0 = pyhd3eb1b0_1-isodate = 0.6.0 = py_1-itsdangerous = 1.1.0 = pyhd3eb1b0_0-絕地= 0.17.2 = py38h06a4308_1-jinja2 = 2.11.2 = pyhd3eb1b0_0-jmespath = 0.10.0 = py_0-joblib = 0.17.0 = py_0-jpeg = 9 b = h024ee3a_2-jupyter_client = 6.1.7 = py_0-jupyter_core = 4.6.3 = py38_0-= py38h2531618_0 1.3.0 kiwisolver =版本-krb5 = 1.17.1 = h173b8e3_0-蹩腳的= 3.100 = h7b6447c_0-lcms2 = 2.11 = h396b838_0-ld_impl_linux - 64 = 2.33.1 = h53a641e_7-libedit = 3.1.20191231 = h14c3975_1-libffi = 3.3 = he6710b0_2-libgcc-ng = 9.1.0 = hdf63c60_0-libgfortran-ng = 7.3.0 = hdf63c60_0-libidn2 = 2.3.1 = h27cfd23_0-libopus = 1.3.1 = h7b6447c_0-libpng = 1.6.37 = hbc83047_0-libpq = 12.2 = h20c2e04_0-libprotobuf = 3.13.0.1 = hd408876_0-libsodium = 1.0.18 = h7b6447c_0-libstdcxx-ng = 9.1.0 = hdf63c60_0-libtasn1 = 4.16.0 = h27cfd23_0-libtiff = 4.1.0 = h2733197_1-libunistring 0.9.10 = = h27cfd23_0-libuv = 1.40.0 = h7b6447c_0-libvpx = 1.7.0 = h439df22_0-lightgbm = 3.1.1 = py38h2531618_0-lz4-c = 1.9.2 = heb0550a_3-尖吻鯖鯊= 1.1.3 = py_0-減價= 3.3.3 = py38h06a4308_0-markupsafe = 1.1.1 = py38h7b6447c_0-matplotlib-base = 3.2.2 = py38hef1b27d_0-mkl = 2019.4 = 243-mkl-service = tripwire = py38he904b0f_0-mkl_fft = 1.2.0 = py38h23d657b_0-mkl_random = 1.1.0 = py38h962f231_0-more-itertools = 8.6.0 = pyhd3eb1b0_0-multidict = 5.1.0 = py38h27cfd23_2-ncurses = 6.2 = he6710b0_1-蕁麻= 3.7.3 = hbbd107a_1-networkx = 2.5.1 = pyhd3eb1b0_0-忍者= 1.10.2 = hff7bd54_1-nltk = 3.5 = py_0-numpy = 1.19.2 = py38h54aff64_0-numpy-base = 1.19.2 = py38hfa32c7d_0-oauthlib = 3.1.0 = py_0-olefile = 0.46 = py_0-openh264 = 2.1.0 = hd408876_0-openssl = 1.1.1k = h27cfd23_0-包裝= 20.4 = py_0-熊貓= 1.1.5 = py38ha9443f7_0-paramiko = 2.7.2 = py_0-parso = 0.7.0 = py_0-容易受騙的人= 0.5.1 = py38_0-pexpect = 4.8.0 = pyhd3eb1b0_3-pickleshare = 0.7.5 = pyhd3eb1b0_1003-枕頭= 8.0.1 = py38he98fc37_0-皮普= 20.2.4 = py38h06a4308_0-情節= 4.14.3 = pyhd3eb1b0_0-prompt-toolkit = 3.0.8 = py_0-prompt_toolkit = 3.0.8 = 0-protobuf = 3.13.0.1 = py38he6710b0_1-psutil = 5.7.2 = py38h7b6447c_0-psycopg2 = 2.8.5 = py38h3c74f83_1-ptyprocess = 0.6.0 = pyhd3eb1b0_2-pyasn1 = 0.4.8 = py_0-pyasn1-modules = 0.2.8 = py_0-pycparser = 2.20 = py_2-pygments = 2.7.2 = pyhd3eb1b0_0-pyjwt = 1.7.1上= py38_0-pynacl = 1.4.0 = py38h7b6447c_1-pyodbc = 4.0.30 = py38he6710b0_0-pyopenssl = 19.1.0 = pyhd3eb1b0_1-pyparsing = 2.4.7 = pyhd3eb1b0_0-pysocks = 1.7.1上= py38h06a4308_0-python = 3.8.8 = hdb3f193_4-python-dateutil = 2.8.1發布= pyhd3eb1b0_0-python編輯器的1.0.4 = = py_0-pytorch = 1.9.0 = py3.8_cpu_0-pytz = 2020.5 = pyhd3eb1b0_0-pyzmq = 19.0.2 = py38he6710b0_1-readline = 8.0 = h7b6447c_0-regex = 2020.10.15 = py38h7b6447c_0-= 2.24.0 = py_0請求-= py_0 1.3.0 requests-oauthlib =版本-重試= 1.3.3 = py_2-rsa = 4.7.2 = pyhd3eb1b0_1-s3transfer = 0.3.6 = pyhd3eb1b0_0-scikit-learn = 0.23.2 = py38h0573a6f_0-scipy = 1.5.2 = py38h0b6359f_0-setuptools = 50.3.1 = py38h06a4308_1-simplejson = 3.17.2 = py38h27cfd23_2-6 = 1.15.0 = py38h06a4308_0-smmap = 3.0.5 = pyhd3eb1b0_0-sqlite = 3.33.0 = h62c20be_0-sqlparse = 0.4.1 = py_0-statsmodels = 0.12.0 = py38h7b6447c_0-彙總= 0.8.7 = py38h06a4308_0-threadpoolctl = 2.1.0 = pyh5ca1d4c_0-tk = 8.6.10 = hbc83047_0-torchvision = 0.10.0 = py38_cpu-龍卷風= 6.0.4 = py38h7b6447c_1-tqdm = 4.50.2 = py_0-traitlets = 5.0.5 = pyhd3eb1b0_0-typing-extensions = 3.7.4.3 = hd3eb1b0_0-typing_extensions = 3.7.4.3 = pyh06a4308_0-unixodbc = 2.3.9 = h7b6447c_0-urllib3 = 1.25.11 = py_0-wcwidth = 0.2.5 = py_0-websocket-client = 0.57.0 = py38_2-werkzeug = 1.0.1 = pyhd3eb1b0_0-輪= 0.35.1 = pyhd3eb1b0_0-打包= 1.12.1 = py38h7b6447c_1-157.20191217 = h7b6447c_0 x264 = 1 !-xz = 5.2.5 = h7b6447c_0-yarl = 1.6.3 = py38h27cfd23_0-zeromq = 4.3.3 = he6710b0_3-氧化鋅碘仿糊= 3.4.0 = pyhd3eb1b0_0-zlib = 1.2.11 = h7b6447c_3-zstd = 1.4.5 = h9ceee32_0-皮普:-argon2-cffi = = 20.1.0-astunparse = = 1.6.3-async-generator = = 1.10-azure-core = = 1.11.0-azure-storage-blob = = 12.7.1-漂白劑= = 3.3.0-瓶頸= = 1.3.2-2.3.2 convertdate = =-databricks-cli = = 0.14.3-defusedxml = =是0.7.1-diskcache = = 5.2.1-碼頭工人= = 4.4.4-facets-overview = = 1.0.0-flatbuffers = = 1.12-grpcio = = 1.34.1-h5py = = 3.1.0-hijri-converter = = 2.1.3-假期= = 0.10.5.2-horovod = = 0.22.1-htmlmin = = 0.1.12-imagehash = = 4.2.0-ipywidgets = = 7.6.3-joblibspark = = 0.3.0-jsonschema = = 3.2.0-jupyterlab-pygments = = 0.1.2-jupyterlab-widgets = = 1.0.0-keras-nightly = = 2.5.0.dev2021032900-keras-preprocessing = = 1.1.2-考拉= = 1.8.1-korean-lunar-calendar = = 0.2.1-llvmlite = = 0.36.0-missingno = = 0.4.2-使走調= = 0.8.4-mleap = = 0.17.0-mlflow-skinny = = 1.18.0-msr = = 0.6.21-多重方法= = 1.4-nbclient = = 0.5.3-nbconvert = = 6.1.0-nbformat = = 5.1.3-nest-asyncio = = 1.5.1-筆記本電腦= = 6.4.0-numba = = 0.53.1-opt-einsum = = 3.3.0-pandas-profiling = = 3.0.0-pandocfilters = = 3-petastorm = = 0.11.1-phik = = 0.11.2-prometheus-client = = 0.11.0-pyarrow = = 1.0.1-pydantic = = 1.8.2-pymeeus = = 0.5.11-pyrsistent = = 0.18.0-pywavelets = = 1.1.1-pyyaml = = 5.4.1之前-querystring-parser = = 4-seaborn = = 0.10.0-send2trash = = 1.7.1上-世鵬科技電子= = 0.39.0-切片機= = 0.0.7-spark-tensorflow-distributor = = 0.1.0-tangled-up-in-unicode = = 0.1.0-tensorboard = = 2.5.0-tensorboard-data-server = = 0.6.1-tensorboard-plugin-wit = = 1.8.0-tensorflow-cpu = = 2.5.0-tensorflow-estimator = = 2.5.0-termcolor = = 1.1.0-terminado = = 0.10.1-testpath = = 0.5.0-= =是0.7.1願景-webencodings = = 0.5.1-widgetsnbextension = = 3.5.1-xgboost = = 1.4.2前綴:/磚/ conda / env / databricks-ml
GPU集群上的Python庫
名字:databricks-ml-gpu渠道:-違約依賴關係:-_libgcc_mutex = 0.1 =主要-absl-py = 0.11.0 = pyhd3eb1b0_1它-aiohttp = 3.7.4 = py38h27cfd23_1-asn1crypto = 1.4.0 = py_0-阿斯特= 0.8.1 = py38h06a4308_0-async-timeout = 3.0.1 = py38h06a4308_0-attrs = 20.3.0 = pyhd3eb1b0_0-backcall = 0.2.0 = pyhd3eb1b0_0-bcrypt = 3.2.0 = py38h7b6447c_0-布拉斯特區= 1.0 = mkl-信號燈= 1.4 = py38h06a4308_0-boto3 = 1.16.7 = pyhd3eb1b0_0-botocore = 1.19.7 = pyhd3eb1b0_0-brotlipy = 0.7.0 = py38h27cfd23_1003-ca證書= 2021.5.25 = h06a4308_1-cachetools = 4.2.2 = pyhd3eb1b0_0-certifi = 2021.5.30 = py38h06a4308_0-cffi = 1.14.3 = py38h261ae71_2-chardet = 3.0.4 = py38h06a4308_1003-單擊= 7.1.2 = pyhd3eb1b0_0-cloudpickle = 1.6.0 = py_0-configparser = 5.0.1 = py_0-密碼= 3.1.1 = py38h1ba5d50_0-周期計= 0.10.0 = py38_0-cython = 0.29.21 = py38h2531618_0-decorator = 10/24/11 = pyhd3eb1b0_0-蒔蘿= 0.3.2 = py_0-docutils = 0.15.2 = py38h06a4308_1-entrypoints = 0.3 = py38_0-瓶= 1.1.2 = pyhd3eb1b0_0-freetype的= 2.10.4 = h5ab3b9f_0-fsspec = 0.8.3 = py_0-未來= 0.18.2 = py38_1-恐嚇= 0.4.0 = py_0-gitdb = 4.0.7 = pyhd3eb1b0_0-gitpython = 3.1.12 = pyhd3eb1b0_1-google-auth = 1.22.1 = py_0-google-auth-oauthlib = 0.4.2 = pyhd3eb1b0_2-google-pasta = 0.2.0 = py_0-gunicorn = 20.0.4 = py38h06a4308_0-hdf5 = 1.10.4 = hb1b8bf9_0-icu = 58.2 = he6710b0_3-idna = 2.10 = pyhd3eb1b0_0-importlib-metadata = 2.0.0 = py_1-intel-openmp = 2019.4 = 243-ipykernel = 5.3.4 = py38h5ca1d4c_0-ipython = 7.19.0 = py38hb070fc8_1-ipython_genutils = 0.2.0 = pyhd3eb1b0_1-isodate = 0.6.0 = py_1-itsdangerous = 1.1.0 = pyhd3eb1b0_0-絕地= 0.17.2 = py38h06a4308_1-jinja2 = 2.11.2 = pyhd3eb1b0_0-jmespath = 0.10.0 = py_0-joblib = 0.17.0 = py_0-jpeg = 9 b = h024ee3a_2-jupyter_client = 6.1.7 = py_0-jupyter_core = 4.6.3 = py38_0-= py38h2531618_0 1.3.0 kiwisolver =版本-krb5 = 1.17.1 = h173b8e3_0-lcms2 = 2.11 = h396b838_0-ld_impl_linux - 64 = 2.33.1 = h53a641e_7-libedit = 3.1.20191231 = h14c3975_1-libffi = 3.3 = he6710b0_2-libgcc-ng = 9.1.0 = hdf63c60_0-libgfortran-ng = 7.3.0 = hdf63c60_0-libpng = 1.6.37 = hbc83047_0-libpq = 12.2 = h20c2e04_0-libprotobuf = 3.13.0.1 = hd408876_0-libsodium = 1.0.18 = h7b6447c_0-libstdcxx-ng = 9.1.0 = hdf63c60_0-libtiff = 4.1.0 = h2733197_1-lightgbm = 3.1.1 = py38h2531618_0-lz4-c = 1.9.2 = heb0550a_3-尖吻鯖鯊= 1.1.3 = py_0-減價= 3.3.3 = py38h06a4308_0-markupsafe = 1.1.1 = py38h7b6447c_0-matplotlib-base = 3.2.2 = py38hef1b27d_0-mkl = 2019.4 = 243-mkl-service = tripwire = py38he904b0f_0-mkl_fft = 1.2.0 = py38h23d657b_0-mkl_random = 1.1.0 = py38h962f231_0-more-itertools = 8.6.0 = pyhd3eb1b0_0-multidict = 5.1.0 = py38h27cfd23_2-ncurses = 6.2 = he6710b0_1-networkx = 2.5.1 = pyhd3eb1b0_0-nltk = 3.5 = py_0-numpy = 1.19.2 = py38h54aff64_0-numpy-base = 1.19.2 = py38hfa32c7d_0-oauthlib = 3.1.0 = py_0-olefile = 0.46 = py_0-openssl = 1.1.1k = h27cfd23_0-包裝= 20.4 = py_0-熊貓= 1.1.5 = py38ha9443f7_0-paramiko = 2.7.2 = py_0-parso = 0.7.0 = py_0-容易受騙的人= 0.5.1 = py38_0-pexpect = 4.8.0 = pyhd3eb1b0_3-pickleshare = 0.7.5 = pyhd3eb1b0_1003-枕頭= 8.0.1 = py38he98fc37_0-皮普= 20.2.4 = py38h06a4308_0-情節= 4.14.3 = pyhd3eb1b0_0-prompt-toolkit = 3.0.8 = py_0-prompt_toolkit = 3.0.8 = 0-protobuf = 3.13.0.1 = py38he6710b0_1-psutil = 5.7.2 = py38h7b6447c_0-psycopg2 = 2.8.5 = py38h3c74f83_1-ptyprocess = 0.6.0 = pyhd3eb1b0_2-pyasn1 = 0.4.8 = py_0-pyasn1-modules = 0.2.8 = py_0-pycparser = 2.20 = py_2-pygments = 2.7.2 = pyhd3eb1b0_0-pyjwt = 1.7.1上= py38_0-pynacl = 1.4.0 = py38h7b6447c_1-pyodbc = 4.0.30 = py38he6710b0_0-pyopenssl = 19.1.0 = pyhd3eb1b0_1-pyparsing = 2.4.7 = pyhd3eb1b0_0-pysocks = 1.7.1上= py38h06a4308_0-python = 3.8.8 = hdb3f193_4-python-dateutil = 2.8.1發布= pyhd3eb1b0_0-python編輯器的1.0.4 = = py_0-pytz = 2020.5 = pyhd3eb1b0_0-pyzmq = 19.0.2 = py38he6710b0_1-readline = 8.0 = h7b6447c_0-regex = 2020.10.15 = py38h7b6447c_0-= 2.24.0 = py_0請求-= py_0 1.3.0 requests-oauthlib =版本-重試= 1.3.3 = py_2-rsa = 4.7.2 = pyhd3eb1b0_1-s3transfer = 0.3.6 = pyhd3eb1b0_0-scikit-learn = 0.23.2 = py38h0573a6f_0-scipy = 1.5.2 = py38h0b6359f_0-setuptools = 50.3.1 = py38h06a4308_1-simplejson = 3.17.2 = py38h27cfd23_2-6 = 1.15.0 = py38h06a4308_0-smmap = 3.0.5 = pyhd3eb1b0_0-sqlite = 3.33.0 = h62c20be_0-sqlparse = 0.4.1 = py_0-statsmodels = 0.12.0 = py38h7b6447c_0-彙總= 0.8.7 = py38h06a4308_0-threadpoolctl = 2.1.0 = pyh5ca1d4c_0-tk = 8.6.10 = hbc83047_0-龍卷風= 6.0.4 = py38h7b6447c_1-tqdm = 4.50.2 = py_0-traitlets = 5.0.5 = pyhd3eb1b0_0-typing-extensions = 3.7.4.3 = hd3eb1b0_0-typing_extensions = 3.7.4.3 = pyh06a4308_0-unixodbc = 2.3.9 = h7b6447c_0-urllib3 = 1.25.11 = py_0-wcwidth = 0.2.5 = py_0-websocket-client = 0.57.0 = py38_2-werkzeug = 1.0.1 = pyhd3eb1b0_0-輪= 0.35.1 = pyhd3eb1b0_0-打包= 1.12.1 = py38h7b6447c_1-xz = 5.2.5 = h7b6447c_0-yarl = 1.6.3 = py38h27cfd23_0-zeromq = 4.3.3 = he6710b0_3-氧化鋅碘仿糊= 3.4.0 = pyhd3eb1b0_0-zlib = 1.2.11 = h7b6447c_3-zstd = 1.4.5 = h9ceee32_0-皮普:-argon2-cffi = = 20.1.0-astunparse = = 1.6.3-async-generator = = 1.10-azure-core = = 1.11.0-azure-storage-blob = = 12.7.1-漂白劑= = 3.3.0-瓶頸= = 1.3.2-2.3.2 convertdate = =-databricks-cli = = 0.14.3-defusedxml = =是0.7.1-diskcache = = 5.2.1-碼頭工人= = 4.4.4-facets-overview = = 1.0.0-flatbuffers = = 1.12-grpcio = = 1.34.1-h5py = = 3.1.0-hijri-converter = = 2.1.3-假期= = 0.10.5.2-horovod = = 0.22.1-htmlmin = = 0.1.12-imagehash = = 4.2.0-ipywidgets = = 7.6.3-joblibspark = = 0.3.0-jsonschema = = 3.2.0-jupyterlab-pygments = = 0.1.2-jupyterlab-widgets = = 1.0.0-keras-nightly = = 2.5.0.dev2021032900-keras-preprocessing = = 1.1.2-考拉= = 1.8.1-korean-lunar-calendar = = 0.2.1-llvmlite = = 0.36.0-missingno = = 0.4.2-使走調= = 0.8.4-mleap = = 0.17.0-mlflow-skinny = = 1.18.0-msr = = 0.6.21-多重方法= = 1.4-nbclient = = 0.5.3-nbconvert = = 6.1.0-nbformat = = 5.1.3-nest-asyncio = = 1.5.1-筆記本電腦= = 6.4.0-numba = = 0.53.1-opt-einsum = = 3.3.0-pandas-profiling = = 3.0.0-pandocfilters = = 3-petastorm = = 0.11.1-phik = = 0.11.2-pyarrow = = 1.0.1-pydantic = = 1.8.2-pymeeus = = 0.5.11-pyrsistent = = 0.17.3-pywavelets = = 1.1.1-pyyaml = = 5.4.1之前-querystring-parser = = 4-seaborn = = 0.10.0-send2trash = = 1.7.1上-世鵬科技電子= = 0.39.0-切片機= = 0.0.7-spark-tensorflow-distributor = = 0.1.0-tangled-up-in-unicode = = 0.1.0-tensorboard = = 2.5.0-tensorboard-data-server = = 0.6.1-tensorboard-plugin-wit = = 1.8.0-tensorflow = = 2.5.0-tensorflow-estimator = = 2.5.0-termcolor = = 1.1.0-terminado = = 0.10.1-testpath = = 0.5.0-火炬= = 1.9.0-torchvision = = 0.10.0-= =是0.7.1願景-webencodings = = 0.5.1-widgetsnbextension = = 3.5.1-xgboost = = 1.4.2前綴:/磚/ conda / env / databricks-ml-gpu
Java和Scala庫(Scala 2.12集群)
在Databricks Runtime 8.4中,除了Java和Scala庫,Databricks Runtime 8.4 ML還包含以下jar:
CPU集群
組ID |
工件ID |
版本 |
---|---|---|
com.typesafe.akka |
akka-actor_2.12 |
2.5.23 |
ml.combust.mleap |
mleap-databricks-runtime_2.12 |
0.17.3-4882dc3 |
ml.dmlc |
xgboost4j-spark_2.12 |
1.4.1 |
ml.dmlc |
xgboost4j_2.12 |
1.4.1 |
org.mlflow |
mlflow-client |
1.18.0 |
org.scala-lang.modules |
scala-java8-compat_2.12 |
0.8.0 |
org.tensorflow |
spark-tensorflow-connector_2.12 |
1.15.0 |
GPU集群
組ID |
工件ID |
版本 |
---|---|---|
com.typesafe.akka |
akka-actor_2.12 |
2.5.23 |
ml.combust.mleap |
mleap-databricks-runtime_2.12 |
0.17.3-4882dc3 |
ml.dmlc |
xgboost4j-spark-gpu_2.12 |
1.4.1 |
ml.dmlc |
xgboost4j-gpu_2.12 |
1.4.1 |
org.mlflow |
mlflow-client |
1.18.0 |
org.scala-lang.modules |
scala-java8-compat_2.12 |
0.8.0 |
org.tensorflow |
spark-tensorflow-connector_2.12 |
1.15.0 |