Databricks Runtime 8.4 for ML(不支持)

Databricks於2021年7月發布了這張圖片。

Databricks Runtime 8.4 for Machine Learning為機器學習和數據科學提供了一個現成的環境Databricks Runtime 8.4(不支持).Databricks Runtime ML包含許多流行的機器學習庫,包括TensorFlow、PyTorch和XGBoost。它還支持使用Horovod進行分布式深度學習訓練。

有關更多信息,包括創建Databricks Runtime ML集群的說明,請參見介紹Databricks運行時機器學習

新功能和改進

Databricks Runtime 8.4 ML是建立在Databricks Runtime 8.4之上的。有關Databricks Runtime 8.4(包括Apache Spark MLlib和SparkR)中的新特性的信息,請參見Databricks Runtime 8.4(不支持)發行說明。

FeatureStoreClient v0.3.2

  • 允許特性和特性表名稱與SQL保留字衝突。

  • 驗證所提供的數據幀是否為PySpark數據幀(pyspark.sql.dataframe.DataFrame).

AutoML v1.1.0

  • Databricks Runtime 8.4 ML附帶的AutoML更新版本包括一些錯誤修複和穩定性改進。

  • AutoML分類現在也運行試驗LGBMClassifier

  • AutoML回歸現在也運行試驗LGBMRegressor

Databricks Runtime ML Python環境的主要更改

看到Databricks Runtime 8.4(不支持)參閱Databricks Runtime Python環境的主要更改。有關已安裝的Python包及其版本的完整列表,請參見Python庫

Python包升級

  • 考拉1.8.0 -> 1.8.1

  • Horovod 0.21.3 -> 0.22.1

  • Mleap 0.16.1 -> 0.17.0

  • Mlflow 1.16.0 -> 1.18.0

  • panda -profiling 2.11.0 -> 3.0.0

  • 風暴0.10.0 -> 0.11.1

  • Pytorch 1.8.1 -> 1.9.0

  • Tensorboard 2.4.1 -> 2.5.0

  • Tensorflow 2.4.1 -> 2.5.0

  • Torchvision 0.9.1 -> 0.10.0

  • Xgboost 1.4.1 -> 1.4.2

的用法

以下更改已棄用,並將在Databricks Runtime 9.0中刪除:

  • 在HorovodRunner中,設置np = 0,在那裏np是用於Horovod作業的並行進程數。

  • 英特爾數學內核庫(Intel MKL),以及依賴它的下遊包。

  • azure-corepython庫用於核心Azure異常和模塊

  • azure-storage-blobpython庫客戶端用於與Azure Storage Blob服務交互

  • msrpython庫用於AutoRest swagger生成

  • 碼頭工人Docker引擎API的python庫

  • querystring-parserpython庫用於在python /Django中解析查詢

  • intel-openmpPython庫,用於創建多線程軟件

係統環境

Databricks Runtime 8.4 ML的係統環境與Databricks Runtime 8.4不同:

以下部分列出了Databricks Runtime 8.4 ML中包含的不同於Databricks Runtime 8.4的庫。

Python庫

Databricks Runtime 8.4 ML使用Conda進行Python包管理,包括許多流行的ML包。

除了以下章節中Conda環境中指定的包外,Databricks Runtime 8.4 ML還包括以下包:

  • hyperopt 0.2.5.db2

  • sparkdl 2.1.0.db4

  • feature_store 0.3.2

  • automl 1.1.0

CPU集群上的Python庫

名字databricks-ml渠道-pytorch-違約依賴關係-_libgcc_mutex = 0.1 =主要-absl-py = 0.11.0 = pyhd3eb1b0_1它-aiohttp = 3.7.4 = py38h27cfd23_1-asn1crypto = 1.4.0 = py_0-阿斯特= 0.8.1 = py38h06a4308_0-async-timeout = 3.0.1 = py38h06a4308_0-attrs = 20.3.0 = pyhd3eb1b0_0-backcall = 0.2.0 = pyhd3eb1b0_0-bcrypt = 3.2.0 = py38h7b6447c_0-布拉斯特區= 1.0 = mkl-信號燈= 1.4 = py38h06a4308_0-boto3 = 1.16.7 = pyhd3eb1b0_0-botocore = 1.19.7 = pyhd3eb1b0_0-brotlipy = 0.7.0 = py38h27cfd23_1003-bzip2 = 1.0.8 = h7b6447c_0-ca證書= 2021.5.25 = h06a4308_1-cachetools = 4.2.2 = pyhd3eb1b0_0-certifi = 2021.5.30 = py38h06a4308_0-cffi = 1.14.3 = py38h261ae71_2-chardet = 3.0.4 = py38h06a4308_1003-單擊= 7.1.2 = pyhd3eb1b0_0-cloudpickle = 1.6.0 = py_0-configparser = 5.0.1 = py_0-cpuonly = 1.0 = 0-密碼= 3.1.1 = py38h1ba5d50_0-周期計= 0.10.0 = py38_0-cython = 0.29.21 = py38h2531618_0-decorator = 10/24/11 = pyhd3eb1b0_0-蒔蘿= 0.3.2 = py_0-docutils = 0.15.2 = py38h06a4308_1-entrypoints = 0.3 = py38_0-ffmpeg = 4.2.2 = h20bf706_0-瓶= 1.1.2 = pyhd3eb1b0_0-freetype的= 2.10.4 = h5ab3b9f_0-fsspec = 0.8.3 = py_0-未來= 0.18.2 = py38_1-恐嚇= 0.4.0 = py_0-gitdb = 4.0.7 = pyhd3eb1b0_0-gitpython = 3.1.12 = pyhd3eb1b0_1-6.1.2 gmp = = h6c8ec71_1-gnutls = 3.6.15 = he1e5248_0-google-auth = 1.22.1 = py_0-google-auth-oauthlib = 0.4.2 = pyhd3eb1b0_2-google-pasta = 0.2.0 = py_0-gunicorn = 20.0.4 = py38h06a4308_0-hdf5 = 1.10.4 = hb1b8bf9_0-icu = 58.2 = he6710b0_3-idna = 2.10 = pyhd3eb1b0_0-importlib-metadata = 2.0.0 = py_1-intel-openmp = 2019.4 = 243-ipykernel = 5.3.4 = py38h5ca1d4c_0-ipython = 7.19.0 = py38hb070fc8_1-ipython_genutils = 0.2.0 = pyhd3eb1b0_1-isodate = 0.6.0 = py_1-itsdangerous = 1.1.0 = pyhd3eb1b0_0-絕地= 0.17.2 = py38h06a4308_1-jinja2 = 2.11.2 = pyhd3eb1b0_0-jmespath = 0.10.0 = py_0-joblib = 0.17.0 = py_0-jpeg = 9 b = h024ee3a_2-jupyter_client = 6.1.7 = py_0-jupyter_core = 4.6.3 = py38_0-= py38h2531618_0 1.3.0 kiwisolver =版本-krb5 = 1.17.1 = h173b8e3_0-蹩腳的= 3.100 = h7b6447c_0-lcms2 = 2.11 = h396b838_0-ld_impl_linux - 64 = 2.33.1 = h53a641e_7-libedit = 3.1.20191231 = h14c3975_1-libffi = 3.3 = he6710b0_2-libgcc-ng = 9.1.0 = hdf63c60_0-libgfortran-ng = 7.3.0 = hdf63c60_0-libidn2 = 2.3.1 = h27cfd23_0-libopus = 1.3.1 = h7b6447c_0-libpng = 1.6.37 = hbc83047_0-libpq = 12.2 = h20c2e04_0-libprotobuf = 3.13.0.1 = hd408876_0-libsodium = 1.0.18 = h7b6447c_0-libstdcxx-ng = 9.1.0 = hdf63c60_0-libtasn1 = 4.16.0 = h27cfd23_0-libtiff = 4.1.0 = h2733197_1-libunistring 0.9.10 = = h27cfd23_0-libuv = 1.40.0 = h7b6447c_0-libvpx = 1.7.0 = h439df22_0-lightgbm = 3.1.1 = py38h2531618_0-lz4-c = 1.9.2 = heb0550a_3-尖吻鯖鯊= 1.1.3 = py_0-減價= 3.3.3 = py38h06a4308_0-markupsafe = 1.1.1 = py38h7b6447c_0-matplotlib-base = 3.2.2 = py38hef1b27d_0-mkl = 2019.4 = 243-mkl-service = tripwire = py38he904b0f_0-mkl_fft = 1.2.0 = py38h23d657b_0-mkl_random = 1.1.0 = py38h962f231_0-more-itertools = 8.6.0 = pyhd3eb1b0_0-multidict = 5.1.0 = py38h27cfd23_2-ncurses = 6.2 = he6710b0_1-蕁麻= 3.7.3 = hbbd107a_1-networkx = 2.5.1 = pyhd3eb1b0_0-忍者= 1.10.2 = hff7bd54_1-nltk = 3.5 = py_0-numpy = 1.19.2 = py38h54aff64_0-numpy-base = 1.19.2 = py38hfa32c7d_0-oauthlib = 3.1.0 = py_0-olefile = 0.46 = py_0-openh264 = 2.1.0 = hd408876_0-openssl = 1.1.1k = h27cfd23_0-包裝= 20.4 = py_0-熊貓= 1.1.5 = py38ha9443f7_0-paramiko = 2.7.2 = py_0-parso = 0.7.0 = py_0-容易受騙的人= 0.5.1 = py38_0-pexpect = 4.8.0 = pyhd3eb1b0_3-pickleshare = 0.7.5 = pyhd3eb1b0_1003-枕頭= 8.0.1 = py38he98fc37_0-皮普= 20.2.4 = py38h06a4308_0-情節= 4.14.3 = pyhd3eb1b0_0-prompt-toolkit = 3.0.8 = py_0-prompt_toolkit = 3.0.8 = 0-protobuf = 3.13.0.1 = py38he6710b0_1-psutil = 5.7.2 = py38h7b6447c_0-psycopg2 = 2.8.5 = py38h3c74f83_1-ptyprocess = 0.6.0 = pyhd3eb1b0_2-pyasn1 = 0.4.8 = py_0-pyasn1-modules = 0.2.8 = py_0-pycparser = 2.20 = py_2-pygments = 2.7.2 = pyhd3eb1b0_0-pyjwt = 1.7.1上= py38_0-pynacl = 1.4.0 = py38h7b6447c_1-pyodbc = 4.0.30 = py38he6710b0_0-pyopenssl = 19.1.0 = pyhd3eb1b0_1-pyparsing = 2.4.7 = pyhd3eb1b0_0-pysocks = 1.7.1上= py38h06a4308_0-python = 3.8.8 = hdb3f193_4-python-dateutil = 2.8.1發布= pyhd3eb1b0_0-python編輯器的1.0.4 = = py_0-pytorch = 1.9.0 = py3.8_cpu_0-pytz = 2020.5 = pyhd3eb1b0_0-pyzmq = 19.0.2 = py38he6710b0_1-readline = 8.0 = h7b6447c_0-regex = 2020.10.15 = py38h7b6447c_0-= 2.24.0 = py_0請求-= py_0 1.3.0 requests-oauthlib =版本-重試= 1.3.3 = py_2-rsa = 4.7.2 = pyhd3eb1b0_1-s3transfer = 0.3.6 = pyhd3eb1b0_0-scikit-learn = 0.23.2 = py38h0573a6f_0-scipy = 1.5.2 = py38h0b6359f_0-setuptools = 50.3.1 = py38h06a4308_1-simplejson = 3.17.2 = py38h27cfd23_2-6 = 1.15.0 = py38h06a4308_0-smmap = 3.0.5 = pyhd3eb1b0_0-sqlite = 3.33.0 = h62c20be_0-sqlparse = 0.4.1 = py_0-statsmodels = 0.12.0 = py38h7b6447c_0-彙總= 0.8.7 = py38h06a4308_0-threadpoolctl = 2.1.0 = pyh5ca1d4c_0-tk = 8.6.10 = hbc83047_0-torchvision = 0.10.0 = py38_cpu-龍卷風= 6.0.4 = py38h7b6447c_1-tqdm = 4.50.2 = py_0-traitlets = 5.0.5 = pyhd3eb1b0_0-typing-extensions = 3.7.4.3 = hd3eb1b0_0-typing_extensions = 3.7.4.3 = pyh06a4308_0-unixodbc = 2.3.9 = h7b6447c_0-urllib3 = 1.25.11 = py_0-wcwidth = 0.2.5 = py_0-websocket-client = 0.57.0 = py38_2-werkzeug = 1.0.1 = pyhd3eb1b0_0-輪= 0.35.1 = pyhd3eb1b0_0-打包= 1.12.1 = py38h7b6447c_1-157.20191217 = h7b6447c_0 x264 = 1 !-xz = 5.2.5 = h7b6447c_0-yarl = 1.6.3 = py38h27cfd23_0-zeromq = 4.3.3 = he6710b0_3-氧化鋅碘仿糊= 3.4.0 = pyhd3eb1b0_0-zlib = 1.2.11 = h7b6447c_3-zstd = 1.4.5 = h9ceee32_0-皮普-argon2-cffi = = 20.1.0-astunparse = = 1.6.3-async-generator = = 1.10-azure-core = = 1.11.0-azure-storage-blob = = 12.7.1-漂白劑= = 3.3.0-瓶頸= = 1.3.2-2.3.2 convertdate = =-databricks-cli = = 0.14.3-defusedxml = =是0.7.1-diskcache = = 5.2.1-碼頭工人= = 4.4.4-facets-overview = = 1.0.0-flatbuffers = = 1.12-grpcio = = 1.34.1-h5py = = 3.1.0-hijri-converter = = 2.1.3-假期= = 0.10.5.2-horovod = = 0.22.1-htmlmin = = 0.1.12-imagehash = = 4.2.0-ipywidgets = = 7.6.3-joblibspark = = 0.3.0-jsonschema = = 3.2.0-jupyterlab-pygments = = 0.1.2-jupyterlab-widgets = = 1.0.0-keras-nightly = = 2.5.0.dev2021032900-keras-preprocessing = = 1.1.2-考拉= = 1.8.1-korean-lunar-calendar = = 0.2.1-llvmlite = = 0.36.0-missingno = = 0.4.2-使走調= = 0.8.4-mleap = = 0.17.0-mlflow-skinny = = 1.18.0-msr = = 0.6.21-多重方法= = 1.4-nbclient = = 0.5.3-nbconvert = = 6.1.0-nbformat = = 5.1.3-nest-asyncio = = 1.5.1-筆記本電腦= = 6.4.0-numba = = 0.53.1-opt-einsum = = 3.3.0-pandas-profiling = = 3.0.0-pandocfilters = = 3-petastorm = = 0.11.1-phik = = 0.11.2-prometheus-client = = 0.11.0-pyarrow = = 1.0.1-pydantic = = 1.8.2-pymeeus = = 0.5.11-pyrsistent = = 0.18.0-pywavelets = = 1.1.1-pyyaml = = 5.4.1之前-querystring-parser = = 4-seaborn = = 0.10.0-send2trash = = 1.7.1上-世鵬科技電子= = 0.39.0-切片機= = 0.0.7-spark-tensorflow-distributor = = 0.1.0-tangled-up-in-unicode = = 0.1.0-tensorboard = = 2.5.0-tensorboard-data-server = = 0.6.1-tensorboard-plugin-wit = = 1.8.0-tensorflow-cpu = = 2.5.0-tensorflow-estimator = = 2.5.0-termcolor = = 1.1.0-terminado = = 0.10.1-testpath = = 0.5.0-= =是0.7.1願景-webencodings = = 0.5.1-widgetsnbextension = = 3.5.1-xgboost = = 1.4.2前綴/磚/ conda / env / databricks-ml

GPU集群上的Python庫

名字databricks-ml-gpu渠道-違約依賴關係-_libgcc_mutex = 0.1 =主要-absl-py = 0.11.0 = pyhd3eb1b0_1它-aiohttp = 3.7.4 = py38h27cfd23_1-asn1crypto = 1.4.0 = py_0-阿斯特= 0.8.1 = py38h06a4308_0-async-timeout = 3.0.1 = py38h06a4308_0-attrs = 20.3.0 = pyhd3eb1b0_0-backcall = 0.2.0 = pyhd3eb1b0_0-bcrypt = 3.2.0 = py38h7b6447c_0-布拉斯特區= 1.0 = mkl-信號燈= 1.4 = py38h06a4308_0-boto3 = 1.16.7 = pyhd3eb1b0_0-botocore = 1.19.7 = pyhd3eb1b0_0-brotlipy = 0.7.0 = py38h27cfd23_1003-ca證書= 2021.5.25 = h06a4308_1-cachetools = 4.2.2 = pyhd3eb1b0_0-certifi = 2021.5.30 = py38h06a4308_0-cffi = 1.14.3 = py38h261ae71_2-chardet = 3.0.4 = py38h06a4308_1003-單擊= 7.1.2 = pyhd3eb1b0_0-cloudpickle = 1.6.0 = py_0-configparser = 5.0.1 = py_0-密碼= 3.1.1 = py38h1ba5d50_0-周期計= 0.10.0 = py38_0-cython = 0.29.21 = py38h2531618_0-decorator = 10/24/11 = pyhd3eb1b0_0-蒔蘿= 0.3.2 = py_0-docutils = 0.15.2 = py38h06a4308_1-entrypoints = 0.3 = py38_0-瓶= 1.1.2 = pyhd3eb1b0_0-freetype的= 2.10.4 = h5ab3b9f_0-fsspec = 0.8.3 = py_0-未來= 0.18.2 = py38_1-恐嚇= 0.4.0 = py_0-gitdb = 4.0.7 = pyhd3eb1b0_0-gitpython = 3.1.12 = pyhd3eb1b0_1-google-auth = 1.22.1 = py_0-google-auth-oauthlib = 0.4.2 = pyhd3eb1b0_2-google-pasta = 0.2.0 = py_0-gunicorn = 20.0.4 = py38h06a4308_0-hdf5 = 1.10.4 = hb1b8bf9_0-icu = 58.2 = he6710b0_3-idna = 2.10 = pyhd3eb1b0_0-importlib-metadata = 2.0.0 = py_1-intel-openmp = 2019.4 = 243-ipykernel = 5.3.4 = py38h5ca1d4c_0-ipython = 7.19.0 = py38hb070fc8_1-ipython_genutils = 0.2.0 = pyhd3eb1b0_1-isodate = 0.6.0 = py_1-itsdangerous = 1.1.0 = pyhd3eb1b0_0-絕地= 0.17.2 = py38h06a4308_1-jinja2 = 2.11.2 = pyhd3eb1b0_0-jmespath = 0.10.0 = py_0-joblib = 0.17.0 = py_0-jpeg = 9 b = h024ee3a_2-jupyter_client = 6.1.7 = py_0-jupyter_core = 4.6.3 = py38_0-= py38h2531618_0 1.3.0 kiwisolver =版本-krb5 = 1.17.1 = h173b8e3_0-lcms2 = 2.11 = h396b838_0-ld_impl_linux - 64 = 2.33.1 = h53a641e_7-libedit = 3.1.20191231 = h14c3975_1-libffi = 3.3 = he6710b0_2-libgcc-ng = 9.1.0 = hdf63c60_0-libgfortran-ng = 7.3.0 = hdf63c60_0-libpng = 1.6.37 = hbc83047_0-libpq = 12.2 = h20c2e04_0-libprotobuf = 3.13.0.1 = hd408876_0-libsodium = 1.0.18 = h7b6447c_0-libstdcxx-ng = 9.1.0 = hdf63c60_0-libtiff = 4.1.0 = h2733197_1-lightgbm = 3.1.1 = py38h2531618_0-lz4-c = 1.9.2 = heb0550a_3-尖吻鯖鯊= 1.1.3 = py_0-減價= 3.3.3 = py38h06a4308_0-markupsafe = 1.1.1 = py38h7b6447c_0-matplotlib-base = 3.2.2 = py38hef1b27d_0-mkl = 2019.4 = 243-mkl-service = tripwire = py38he904b0f_0-mkl_fft = 1.2.0 = py38h23d657b_0-mkl_random = 1.1.0 = py38h962f231_0-more-itertools = 8.6.0 = pyhd3eb1b0_0-multidict = 5.1.0 = py38h27cfd23_2-ncurses = 6.2 = he6710b0_1-networkx = 2.5.1 = pyhd3eb1b0_0-nltk = 3.5 = py_0-numpy = 1.19.2 = py38h54aff64_0-numpy-base = 1.19.2 = py38hfa32c7d_0-oauthlib = 3.1.0 = py_0-olefile = 0.46 = py_0-openssl = 1.1.1k = h27cfd23_0-包裝= 20.4 = py_0-熊貓= 1.1.5 = py38ha9443f7_0-paramiko = 2.7.2 = py_0-parso = 0.7.0 = py_0-容易受騙的人= 0.5.1 = py38_0-pexpect = 4.8.0 = pyhd3eb1b0_3-pickleshare = 0.7.5 = pyhd3eb1b0_1003-枕頭= 8.0.1 = py38he98fc37_0-皮普= 20.2.4 = py38h06a4308_0-情節= 4.14.3 = pyhd3eb1b0_0-prompt-toolkit = 3.0.8 = py_0-prompt_toolkit = 3.0.8 = 0-protobuf = 3.13.0.1 = py38he6710b0_1-psutil = 5.7.2 = py38h7b6447c_0-psycopg2 = 2.8.5 = py38h3c74f83_1-ptyprocess = 0.6.0 = pyhd3eb1b0_2-pyasn1 = 0.4.8 = py_0-pyasn1-modules = 0.2.8 = py_0-pycparser = 2.20 = py_2-pygments = 2.7.2 = pyhd3eb1b0_0-pyjwt = 1.7.1上= py38_0-pynacl = 1.4.0 = py38h7b6447c_1-pyodbc = 4.0.30 = py38he6710b0_0-pyopenssl = 19.1.0 = pyhd3eb1b0_1-pyparsing = 2.4.7 = pyhd3eb1b0_0-pysocks = 1.7.1上= py38h06a4308_0-python = 3.8.8 = hdb3f193_4-python-dateutil = 2.8.1發布= pyhd3eb1b0_0-python編輯器的1.0.4 = = py_0-pytz = 2020.5 = pyhd3eb1b0_0-pyzmq = 19.0.2 = py38he6710b0_1-readline = 8.0 = h7b6447c_0-regex = 2020.10.15 = py38h7b6447c_0-= 2.24.0 = py_0請求-= py_0 1.3.0 requests-oauthlib =版本-重試= 1.3.3 = py_2-rsa = 4.7.2 = pyhd3eb1b0_1-s3transfer = 0.3.6 = pyhd3eb1b0_0-scikit-learn = 0.23.2 = py38h0573a6f_0-scipy = 1.5.2 = py38h0b6359f_0-setuptools = 50.3.1 = py38h06a4308_1-simplejson = 3.17.2 = py38h27cfd23_2-6 = 1.15.0 = py38h06a4308_0-smmap = 3.0.5 = pyhd3eb1b0_0-sqlite = 3.33.0 = h62c20be_0-sqlparse = 0.4.1 = py_0-statsmodels = 0.12.0 = py38h7b6447c_0-彙總= 0.8.7 = py38h06a4308_0-threadpoolctl = 2.1.0 = pyh5ca1d4c_0-tk = 8.6.10 = hbc83047_0-龍卷風= 6.0.4 = py38h7b6447c_1-tqdm = 4.50.2 = py_0-traitlets = 5.0.5 = pyhd3eb1b0_0-typing-extensions = 3.7.4.3 = hd3eb1b0_0-typing_extensions = 3.7.4.3 = pyh06a4308_0-unixodbc = 2.3.9 = h7b6447c_0-urllib3 = 1.25.11 = py_0-wcwidth = 0.2.5 = py_0-websocket-client = 0.57.0 = py38_2-werkzeug = 1.0.1 = pyhd3eb1b0_0-輪= 0.35.1 = pyhd3eb1b0_0-打包= 1.12.1 = py38h7b6447c_1-xz = 5.2.5 = h7b6447c_0-yarl = 1.6.3 = py38h27cfd23_0-zeromq = 4.3.3 = he6710b0_3-氧化鋅碘仿糊= 3.4.0 = pyhd3eb1b0_0-zlib = 1.2.11 = h7b6447c_3-zstd = 1.4.5 = h9ceee32_0-皮普-argon2-cffi = = 20.1.0-astunparse = = 1.6.3-async-generator = = 1.10-azure-core = = 1.11.0-azure-storage-blob = = 12.7.1-漂白劑= = 3.3.0-瓶頸= = 1.3.2-2.3.2 convertdate = =-databricks-cli = = 0.14.3-defusedxml = =是0.7.1-diskcache = = 5.2.1-碼頭工人= = 4.4.4-facets-overview = = 1.0.0-flatbuffers = = 1.12-grpcio = = 1.34.1-h5py = = 3.1.0-hijri-converter = = 2.1.3-假期= = 0.10.5.2-horovod = = 0.22.1-htmlmin = = 0.1.12-imagehash = = 4.2.0-ipywidgets = = 7.6.3-joblibspark = = 0.3.0-jsonschema = = 3.2.0-jupyterlab-pygments = = 0.1.2-jupyterlab-widgets = = 1.0.0-keras-nightly = = 2.5.0.dev2021032900-keras-preprocessing = = 1.1.2-考拉= = 1.8.1-korean-lunar-calendar = = 0.2.1-llvmlite = = 0.36.0-missingno = = 0.4.2-使走調= = 0.8.4-mleap = = 0.17.0-mlflow-skinny = = 1.18.0-msr = = 0.6.21-多重方法= = 1.4-nbclient = = 0.5.3-nbconvert = = 6.1.0-nbformat = = 5.1.3-nest-asyncio = = 1.5.1-筆記本電腦= = 6.4.0-numba = = 0.53.1-opt-einsum = = 3.3.0-pandas-profiling = = 3.0.0-pandocfilters = = 3-petastorm = = 0.11.1-phik = = 0.11.2-pyarrow = = 1.0.1-pydantic = = 1.8.2-pymeeus = = 0.5.11-pyrsistent = = 0.17.3-pywavelets = = 1.1.1-pyyaml = = 5.4.1之前-querystring-parser = = 4-seaborn = = 0.10.0-send2trash = = 1.7.1上-世鵬科技電子= = 0.39.0-切片機= = 0.0.7-spark-tensorflow-distributor = = 0.1.0-tangled-up-in-unicode = = 0.1.0-tensorboard = = 2.5.0-tensorboard-data-server = = 0.6.1-tensorboard-plugin-wit = = 1.8.0-tensorflow = = 2.5.0-tensorflow-estimator = = 2.5.0-termcolor = = 1.1.0-terminado = = 0.10.1-testpath = = 0.5.0-火炬= = 1.9.0-torchvision = = 0.10.0-= =是0.7.1願景-webencodings = = 0.5.1-widgetsnbextension = = 3.5.1-xgboost = = 1.4.2前綴/磚/ conda / env / databricks-ml-gpu

包含Python模塊的Spark包

火花包

Python模塊

版本

graphframes

graphframes

0.8.1-db3-spark3.1

R庫

R庫與R庫在Databricks Runtime 8.4。

Java和Scala庫(Scala 2.12集群)

在Databricks Runtime 8.4中,除了Java和Scala庫,Databricks Runtime 8.4 ML還包含以下jar:

CPU集群

組ID

工件ID

版本

com.typesafe.akka

akka-actor_2.12

2.5.23

ml.combust.mleap

mleap-databricks-runtime_2.12

0.17.3-4882dc3

ml.dmlc

xgboost4j-spark_2.12

1.4.1

ml.dmlc

xgboost4j_2.12

1.4.1

org.mlflow

mlflow-client

1.18.0

org.scala-lang.modules

scala-java8-compat_2.12

0.8.0

org.tensorflow

spark-tensorflow-connector_2.12

1.15.0

GPU集群

組ID

工件ID

版本

com.typesafe.akka

akka-actor_2.12

2.5.23

ml.combust.mleap

mleap-databricks-runtime_2.12

0.17.3-4882dc3

ml.dmlc

xgboost4j-spark-gpu_2.12

1.4.1

ml.dmlc

xgboost4j-gpu_2.12

1.4.1

org.mlflow

mlflow-client

1.18.0

org.scala-lang.modules

scala-java8-compat_2.12

0.8.0

org.tensorflow

spark-tensorflow-connector_2.12

1.15.0