-:jbD? dZddlZddlmZddlZddlmZmZddlm Z m Z ddl m Z m Z mZmZmZmZmZmZmZmZddlmZed Zed Zeeej6d fZeedzej:d fZeedzej:d fZejAdddg djdededdfdZ!ejAddgdddddddddddddddddddddedeeej:d fdeeej:d fdeeej:d fd!eeej:d" fd#eeej:d$ fd%eeej:d& fd'eeej:d( fded)eedzej:d* fded+ee"ej:d, fd-ee"ej:d. fd/ee"dzej:d0 fd1eedzej:d2 fd3ee#dzej:d4 fd5ed6eedzej:d7d8 fd9eedzej:d: fd;ee"dzej:d< fd=eedzej:d>d? fd@eedzej:dAdB fdCe dDe dEedFedGeedzej:dHeedIJfddf8dKZ$ejAddLg dkdeeej:d fdedeedzej:dM fdededdf dNZ% dldeddfdOZ&ejAddPge&ejAdQdRSe&ejOedTUejAdVg djdedededdfdWZ(ejAdXg dmdededeedzej:d fdeedzej:d fd!eedzej:d" fd#eedzej:d$ fdeedzej:dY fd5eedzej:dZ fd)eedzej:d* fd+ee"dzej:d, fd-ee"dzej:d. fd/ee"dzej:d0 fd1eedzej:d2 fd3ee#dzej:d4 fdeddf d[Z)ejAd\g dndeded]ee*ej:d^d_ fdeddf d`Z+ejAdag djdedededdfdbZ,ejAdcg dndededdee*ej:dedf fdeddf dgZ-ejAdhg djdedededdfdiZ.y)oz2CLI commands for Hugging Face Inference Endpoints.N) Annotated)InferenceEndpointScalingMetricInferenceEndpointType)CLIErrorHfHubHTTPError) EnvFileOptEnvOpt RevisionOptSecretsFileOpt SecretsOpt SoftChoiceTokenOpt get_hf_api parse_env_map typer_factory)outz(Manage Hugging Face Inference Endpoints.)helpz.Interact with the Inference Endpoints catalog.zEndpoint name.z_The namespace associated with the Inference Endpoint. Defaults to the current user's namespace.z list | lszhf endpoints lsz"hf endpoints ls --namespace my-org)examples namespacetokenreturnct|} |j||}g}|D]o}|j}|jdi}|jdi} |jdi} |jd i} |j|jd d t| tr| jd d nd t|tr|jd d nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd drtj|d y#t$rH}tjd|t j |jj|d}~wwxYw)z6Lists all Inference Endpoints for the given namespace.r)rrzListing failed: codeNstatusmodelcomputeprovidername repositorystatetask framework instanceTypevendorregion)r!rrr%r&instancer(r))id_key)rlist_inference_endpointsrrerrortyperExitresponse status_coderawgetappend isinstancedicttable) rrapi endpointsr-resultsendpointr2rrrr s i/var/www/html/tokenscope/api/venv/lib/python3.12/site-packages/huggingface_hub/cli/inference_endpoints.pylsr=0s 5 !CE009E0R G ll2&$'')R(77:r*+8B5$8O<4UW5?5M&**Wb1SU1;E41H &"-b;EeT;RUYY{B7XZ?I'SW?XGKK;^`8B8T8R(,,x4XZ8B8T8R(,,x4XZ   $IIgf%/ E $UG,-jjenn889uDEsF00 H9AG<<HdeployzChf endpoints deploy my-endpoint --repo gpt2 --framework pytorch ...)r!r)rr%r min_replica max_replicascale_to_zero_timeoutscaling_metricscaling_thresholdrevision custom_image health_routeportcontainer_commandcontainer_argsenvenv_filesecrets secrets_file endpoint_typer!repozeThe name of the model repository associated with the Inference Endpoint (e.g. 'openai/gpt-oss-120b').r&z@The machine learning framework used for the model (e.g. 'vllm'). acceleratorz?The hardware accelerator to be used for inference (e.g. 'cpu'). instance_sizezNThe size or type of the instance to be used for hosting the model (e.g. 'x4'). instance_typezYThe cloud instance type where the Inference Endpoint will be deployed (e.g. 'intel-icl').r)zTThe cloud region in which the Inference Endpoint will be created (e.g. 'us-east-1').r(zVThe cloud provider or vendor where the Inference Endpoint will be hosted (e.g. 'aws').r%zCThe task on which to deploy the model (e.g. 'text-classification').r?zVThe minimum number of replicas (instances) to keep running for the Inference Endpoint.r@zRThe maximum number of replicas (instances) to scale to for the Inference Endpoint.rAzFThe duration in minutes before an inactive endpoint is scaled to zero.rBz!The metric reference for scaling.rCzeThe scaling metric threshold used to trigger a scale up. Ignored when scaling metric is not provided.rDrEz--custom-imagezfDocker image URL for a custom container (e.g. 'nexagi/sglang:v0.5.12'). Requires '--framework custom'.rFz]Health check route exposed by the custom container (e.g. '/health'). Requires --custom-image.rGzKPort the custom container listens on (e.g. 30000). Requires --custom-image.rHz--container-commandzOverride the container entrypoint, as a quoted string split into tokens (e.g. "python -m sglang.launch_server"). Requires --custom-image.rIz--container-argszArguments appended to the container entrypoint, as a quoted string split into tokens (e.g. "--tp 8 --reasoning-parser qwen3"). Requires --custom-image.rJrKrLrMrNz--typezTEndpoint access type. Defaults to 'authenticated' (token-gated, publicly reachable).) click_typerc||||s|r tdd}|d|i}|||d<|||d<t||jDcic] \}}||xsd}}}t||jDcic] \}}||xsd}}}i} ||| d<||| d<|rtj|| d <|rtj|| d <|r|| d <|r|| d <t | }!|!j d!id|d|d|d|d|d|d|d|d|d| d| d| d| d|d|d| d|| }"tj|"jtjd|d ycc}}wcc}}w)"z3Deploy an Inference Endpoint from a Hub repository.NzX--health-route, --port, --container-command and --container-args require --custom-image.url healthRouterGr"typerErHrIrJrLrr!r#r&rPrQrRr)r(rr%rr?r@rBrCrArDzUse 'hf endpoints describe z!' to check the deployment status.) rritemsshlexsplitrcreate_inference_endpointrr6r2hint)#r!rOr&rPrQrRr)r(rr%rr?r@rArBrCrDrErFrGrHrIrJrKrLrMrNcustom_image_dictkeyvalueenv_map secrets_mapparamsr8r;s# r<r>r>Ss&V!9T=MQbftqrr%)"L1  #/; m ,  (, f %2?X2N2T2T2VWJCsEKRWGW6CG\6Z6`6`6bc U3 #cKcF &v$!2~&+kk2C&D"##(;;~#> u 'y 5 !C,s,,   $  $   &, 4!" %H(HHX\\HH*4&0Q RSQXcs E*:E0zChf endpoints catalog deploy --repo meta-llama/Llama-3.2-1B-InstructzPThe hardware accelerator to be used for inference (e.g. 'cpu', 'gpu', 'neuron').c0t|} |j|||||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)z4Deploy an Inference Endpoint from the Model Catalog.r)repo_idr!rPrrzDeployment failed: rN) r&create_inference_endpoint_from_catalogrrr-r.r/r0r1r6r2)rOr!rPrrr8r;r-s r<deploy_from_catalogrgs& 5 !C E==# > HHX\\ E 'w/0jjenn889uDEA B ABBct|} |j|}tjd|iy#t$rH}tjd|t j |jj|d}~wwxYw)zList available Catalog models.rzCatalog fetch failed: rNmodels) rlist_inference_catalogrrr-r.r/r0r1r6)rr8rjr-s r< list_catalogrl6s} 5 !CE++%+8 HHh  E *5'23jjenn889uDEs8 B ABB zhf endpoints catalog lsz list-catalogT)r!hiddencatalogr!z!hf endpoints describe my-endpointc,t|} |j|||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)z+Get information about an existing endpoint.rr!rrzFetch failed: rN) rget_inference_endpointrrr-r.r/r0r1r6r2r!rrr8r;r-s r<describertKs 5 !CE--49TY-Z HHX\\ E N5'*+jjenn889uDEA B ABBz/hf endpoints update my-endpoint --min-replica 2zBThe machine learning framework used for the model (e.g. 'custom').zrThe specific model revision to deploy on the Inference Endpoint (e.g. '6c0e6080953db56375760c0471a8c5f2929baf11').cDt|} |j|||||||||| | | | | |}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)zUpdate an existing endpoint.r)r!rr#r&rDr%rPrQrRr?r@rArBrCrzUpdate failed: rN) rupdate_inference_endpointrrr-r.r/r0r1r6r2)r!rrOrPrQrRr&rDr%r?r@rArBrCrr8r;r-s r<updaterx\s^ 5 !CE00#''##"7)/1 (HHX\\ E OE7+,jjenn889uDEs A BABBzhf endpoints delete my-endpointyesz--yeszSkip confirmation prompts.cZtjd|d|t|} |j|||tjd |d | y#t$rH}tj d|t j|jj|d}~wwxYw) z)Delete an Inference Endpoint permanently.zDelete endpoint 'z'?)ryrrqzDelete failed: rNz Deleted 'z'.ro) rconfirmrdelete_inference_endpointrr-r.r/r0r1result)r!rryrr8r-s r<deleter~sKK#D6,#6 5 !CE %%49E%R JJ4&#$/ E OE7+,jjenn889uDEsA B*"AB%%B*zhf endpoints pause my-endpointc,t|} |j|||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)zPause an Inference Endpoint.rrqzPause failed: rN) rpause_inference_endpointrrr-r.r/r0r1r6r2rss r<pausers 5 !CE//TYV[/\ HHX\\ E N5'*+jjenn889uDEruzhf endpoints resume my-endpointfail_if_already_runningz--fail-if-already-runningzWIf `True`, the method will raise an error if the Inference Endpoint is already running.c0t|} |j|||| }tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)zResume an Inference Endpoint.r)r!rr running_okzResume failed: rN) rresume_inference_endpointrrr-r.r/r0r1r6r2)r!rrrr8r;r-s r<resumers 5 !C E0022 1 HHX\\ E OE7+,jjenn889uDErhz&hf endpoints scale-to-zero my-endpointc,t|} |j|||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)z$Scale an Inference Endpoint to zero.rrqzScale To Zero failed: rN) r scale_to_zero_inference_endpointrrr-r.r/r0r1r6r2rss r< scale_to_zerors 5 !CE77TY^c7d HHX\\ E *5'23jjenn889uDEru)NN)NNNN)N)NNNNNNNNNNNNNN)NFN)/__doc__rZtypingrr.$huggingface_hub._inference_endpointsrrhuggingface_hub.errorsrr _cli_utilsr r r r r rrrrr_outputrie_cli catalog_appstrArgumentNameArgOptionNameOpt NamespaceOptcommandr=intfloatr>rgrl add_typerrtrxboolr~rrrrXr<rsB 8  f;    F G!QR  ENN()+  $JELL&')  $JELL n  '8:^&_`"&& & &a&DX)n(op\#      #' M|T |T   x   |T  S  |T  R  |T(  a  )|T4  l  5|T@   g   A|TL   i   M|TZ[|T\  d  V   ]|Th i|Tj  i  k|Tv  e  w|TB% d  Y  C|TN&- 4  O|TZ!   x  [|Tfg|Th d  y  i|Tv d  p  w|TB  d  ^   C|TN! d  !T   O|Tb d  U   c|Tv w|Txy|Tz{|T|!}|T~ d  !"78g  |TN O|Tq|T~ (.s-tu "   x     d  c     !vF !  !  !L 0I/JKLY0N40>9-=>?#       @  KLM# Wd dd  d  x   d d  R  d d  a  d* d  l  +d6 d  U  7dB d  F  CdN  d  V   OdZ d  i  [df d  e  gdr% d  Y  sd~&- 4  dJ!   x  KdV WdX YdNdN;<=# 0 00   W#?@ B 0 0 0>0,:;<#       =  ;<=#  '  'j    >6BCD#       E r