集群CLI(遺留)
重要的
這個文檔已經退休了,可能不會被更新。
這個信息適用於遺留數據磚CLI版本0.17及以下。磚建議你使用新磚CLI的版本0.200或更高版本。看到磚CLI。找到你的版本的磚CLI,運行磚- v。
從磚CLI或低於0.17版本遷移到磚CLI版本0.200或更高版本,看看磚CLI的遷移。
你跑磚集群CLI子命令通過添加磚集群。這些子命令調用集群API。
磚集群- h
用法:磚集群[選項]命令(ARGS)……效用與磚集群。選項:- v、——版本(版本)- h,幫助顯示此消息並退出。命令:創建集群創建一個磚。選項:- JSON文件路徑包含JSON請求的文件發布. . / api / 2.0 /集群/創建。——json json json字符串發布. . / api / 2.0 /集群/創建。刪除刪除一個磚集群。選項:——cluster-id CLUSTER_ID可以在URL https:// < databricks-instance > / ? o = < 16-digit-number > # /設置/集群/ CLUSTER_ID美元/配置。編輯編輯一個磚集群。選項:- JSON文件路徑包含JSON請求的文件發布. . / api / 2.0 /集群/編輯。 --json JSON JSON string to POST to ../api/2.0/clusters/edit. events Gets events for a Spark cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///#/setting/clusters/$CLUSTER_ID/configuration. [required] --start-time TEXT The start time in epoch milliseconds. If unprovided, returns events starting from the beginning of time. --end-time TEXT The end time in epoch milliseconds. If unprovided, returns events up to the current time --order TEXT The order to list events in; either ASC or DESC. Defaults to DESC (most recent first). --event-type TEXT An event types to filter on (specify multiple event types by passing the --event-type option multiple times). If empty, all event types are returned. --offset TEXT The offset in the result set. Defaults to 0 (no offset). When an offset is specified and the results are requested in descending order, the end_time field is required. --limit TEXT The maximum number of events to include in a page of events. Defaults to 50, and maximum allowed value is 500. --output FORMAT can be "JSON" or "TABLE". Set to TABLE by default. get Retrieves metadata about a cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. list Lists active and recently terminated clusters. Options: --output FORMAT JSON or TABLE. Set to TABLE by default. list-node-types Lists node types for a cluster. list-zones Lists zones where clusters can be created. permanent-delete Permanently deletes a cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. resize Resizes a Databricks cluster given its ID. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. --num-workers INTEGER Number of workers. [required] restart Restarts a Databricks cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. spark-versions Lists possible Databricks Runtime versions. start Starts a terminated Databricks cluster. Options: --cluster-id CLUSTER_ID Can be found in the URL at https:///?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration.
創建一個集群
顯示使用文檔,運行磚集群創建——幫助。
磚集群的創建,json文件create-cluster.json
create-cluster.json:
{“cluster_name”:“my-cluster”,“spark_version”:“7.3.x-scala2.12”,“node_type_id”:“i3.xlarge”,“spark_conf”:{“spark.speculation”:真正的},“aws_attributes”:{“可用性”:“現貨”,“zone_id”:“us-west-2a”},“num_workers”:25}
{“cluster_id”:“1234 - 567890 batch123”}
改變一個集群的配置
顯示使用文檔,運行磚集群編輯——幫助。
磚集群的編輯——json文件edit-cluster.json
edit-cluster.json:
{“cluster_id”:“1234 - 567890 batch123”,“num_workers”:10,“spark_version”:“7.3.x-scala2.12”,“node_type_id”:“i3.xlarge”}
如果成功,沒有顯示輸出。
集群事件列表
顯示使用文檔,運行磚集群事件——幫助。
磚集群事件\——cluster-id1234年567890 - batch123\——啟動時間1617238800000\——世界末日1619485200000\——訂單DESC\——限製5\——事件類型的運行\——JSON輸出\|金橋。
{“事件”:[{“cluster_id”:“1234 - 567890 batch123”,“時間戳”:1619214150232,“類型”:“運行”,"詳細信息":{“current_num_workers”: 2“target_num_workers”: 2}},…{“cluster_id”:“1234 - 567890 batch123”,“時間戳”:1617895221986,“類型”:“運行”,"詳細信息":{“current_num_workers”: 2“target_num_workers”: 2}})," next_page ": {“cluster_id”:“1234 - 567890 batch123”,“start_time”: 1617238800000,“end_time”: 1619485200000,“訂單”:“DESC”,“event_types”:(“運行”),“抵消”:5“限製”:5},“total_count”: 11}
獲得一個集群信息
顯示使用文檔,運行磚集群得到——幫助。
磚集群——cluster-id1234年567890 - batch123
或者:
磚集群,集群名稱my-cluster
{“cluster_id”:“1234 - 567890 batch123”,“spark_context_id”: 8232037838300762810,:“cluster_name my-cluster”,:“spark_version 8.1.x-scala2.12”," aws_attributes ": {:“zone_id us-west-2c”,“first_on_demand”: 1、“可用性”:“SPOT_WITH_FALLBACK”,“spot_bid_price_percent”: 100年,“ebs_volume_count”: 0},:“node_type_id i3.xlarge”,:“driver_node_type_id i3.xlarge”,“autotermination_minutes”: 120年,“enable_elastic_disk”:假的," disk_spec ": {“disk_count”: 0},“cluster_source”:“用戶界麵”,“enable_local_disk_encryption”:假的," instance_source ": {:“node_type_id i3.xlarge”}," driver_instance_source ": {:“node_type_id i3.xlarge”},“狀態”:“終止”,“state_message”:“不活躍集群終止(120分鍾不活躍)。”,“start_time”: 1616773202562,“terminated_time”: 1619228528317,“last_state_loss_time”: 1619214150116,"自動定量":{“min_workers”: 2“max_workers”: 8}," default_tags ": {“供應商”:“磚”,“創造者”:“someone@example.com”,:“ClusterName my-cluster”,“ClusterId”:“1234 - 567890 batch123”},“creator_user_name”:“somone@example.com”," termination_reason ": {“代碼”:“不活躍”,“參數”:{“inactivity_duration_min”:“120”},“類型”:“成功”},“init_scripts_safe_mode”:假的}
列出所有可用的集群信息
顯示使用文檔,運行磚集群列表——幫助。
磚集群列表——JSON輸出|金橋。
{“集群”:({“cluster_id”:“1234 - 567890 batch123”,“spark_context_id”: 8232037838300762810,:“cluster_name my-cluster”,:“spark_version 8.1.x-scala2.12”," aws_attributes ": {:“zone_id us-west-2c”,“first_on_demand”: 1、“可用性”:“SPOT_WITH_FALLBACK”,“spot_bid_price_percent”: 100年,“ebs_volume_count”: 0},:“node_type_id i3.xlarge”,:“driver_node_type_id i3.xlarge”,“autotermination_minutes”: 120年,“enable_elastic_disk”:假的," disk_spec ": {“disk_count”: 0},“cluster_source”:“用戶界麵”,“enable_local_disk_encryption”:假的," instance_source ": {:“node_type_id i3.xlarge”}," driver_instance_source ": {:“node_type_id i3.xlarge”},“狀態”:“終止”,“state_message”:“不活躍集群終止(120分鍾不活躍)。”,“start_time”: 1616773202562,“terminated_time”: 1619228528317,“last_state_loss_time”: 1619214150116,"自動定量":{“min_workers”: 2“max_workers”: 8}," default_tags ": {“供應商”:“磚”,“創造者”:“someone@example.com”,:“ClusterName my-cluster”,“ClusterId”:“1234 - 567890 batch123”},“creator_user_name”:“somone@example.com”," termination_reason ": {“代碼”:“不活躍”,“參數”:{“inactivity_duration_min”:“120”},“類型”:“成功”},“init_scripts_safe_mode”:假的},…]}
可用的集群節點類型列表
顯示使用文檔,運行磚集群list-node-types——幫助。
磚集群list-node-types
{“node_type”:({:“node_type_id z1d.12xlarge”,“memory_mb”: 393216年,“num_cores”: 48.0,“描述”:“z1d.12xlarge”,:“instance_type_id z1d.12xlarge”,“is_deprecated”:假的,“類別”:“內存優化”,“support_ebs_volumes”:沒錯,“support_cluster_tags”:沒錯,“num_gpus”: 0," node_instance_type ": {:“instance_type_id z1d.12xlarge”,“local_disks”: 2“local_disk_size_gb”: 900年,:“instance_family EC2 z1d家庭個vcpu”,“swap_size”:“10 g”},“is_hidden”:假的,“support_port_forwarding”:沒錯,“display_order”: 0,“is_io_cache_enabled”:假的},…]}
列表創建集群的可用區域
顯示使用文檔,運行磚集群list-zones——幫助。
磚集群list-zones
{“區域”:(“us-west-2c”,“us-west-2a”,“us-west-2b”),:“default_zone us-west-2c”}
永久刪除一個集群
顯示使用文檔,運行磚集群permanent-delete——幫助。
磚集群permanent-delete——cluster-id1234年567890 - batch123
如果成功,沒有顯示輸出。