-:jpUddlmZddlmZddlmZddlmZddlm Z d dZ da de d<d d Z dd Zdd Zy)) annotations)cast)Path) Tokenizerc:ttjdz S)Nztokenizer.json)r__file__parentW/var/www/html/tokenscope/api/venv/lib/python3.12/site-packages/anthropic/_tokenizers.py_get_tokenizer_cache_pathr s > #3 33r NzTokenizer | None _tokenizercTtttj|atS)N)rrfrom_strr)raws r _load_tokenizerrs!i!3!3C!89J r chttSt}|jd}t|SNzutf-8)encoding)rr read_textrtokenizer_pathtexts r sync_get_tokenizerrs4.0N  # #W # 5D 4  r cKttStt}|jdd{}t |S7wr)r AsyncPathr rrrs r async_get_tokenizerr%sF8:;N))7); ;D 4   r&s7"# !4 $ #!!r