TiktokenTokenizer: upload tiktoken_encoding.json
Browse files- tiktoken_encoding.json +1 -0
tiktoken_encoding.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"encoding_name": "cl100k_base"}
|