Summarization
Transformers
PyTorch
Safetensors
English
t5
text2text-generation
Trained with AutoTrain
text-generation-inference
Instructions to use sagard21/python-code-explainer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use sagard21/python-code-explainer with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "summarization" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("summarization", model="sagard21/python-code-explainer")# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("sagard21/python-code-explainer") model = AutoModelForSeq2SeqLM.from_pretrained("sagard21/python-code-explainer") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -34,34 +34,12 @@ This model is an attempt to simplify code understanding by generating line by li
|
|
| 34 |
# Model Usage
|
| 35 |
|
| 36 |
```py
|
| 37 |
-
from transformers import AutoTokenizer,
|
| 38 |
|
| 39 |
-
|
| 40 |
-
model=T5ForConditionalGeneration.from_pretrained("sagard21/python-code-explainer"),
|
| 41 |
-
tokenizer=AutoTokenizer.from_pretrained("sagard21/python-code-explainer", skip_special_tokens=True),
|
| 42 |
-
)
|
| 43 |
|
| 44 |
-
|
| 45 |
-
def preprocess(text: str) -> str:
|
| 46 |
-
text = str(text)
|
| 47 |
-
text = text.replace("\n", " ")
|
| 48 |
-
tokenized_text = text.split(" ")
|
| 49 |
-
preprocessed_text = " ".join([token for token in tokenized_text if token])
|
| 50 |
|
| 51 |
-
return preprocessed_text
|
| 52 |
-
"""
|
| 53 |
-
pipeline([raw_code])
|
| 54 |
-
|
| 55 |
-
```
|
| 56 |
-
|
| 57 |
-
### Expected JSON Output
|
| 58 |
-
|
| 59 |
-
```
|
| 60 |
-
[
|
| 61 |
-
{
|
| 62 |
-
"summary_text": "Create a function preprocess that will take the text as an argument and return the preprocessed text.\n1. In this case, the text will be converted to a string.\n2. At first, we will replace all \"\\n\" with \" \" and then split the text by \" \".\n3. Then we will call the tokenize function on the text and tokenize the text using the split() method.\n4. Next step is to create a list of all the tokens in the string and join them together.\n5. Then the function will return the string preprocessed_text.\n"
|
| 63 |
-
}
|
| 64 |
-
]
|
| 65 |
```
|
| 66 |
|
| 67 |
## Validation Metrics
|
|
|
|
| 34 |
# Model Usage
|
| 35 |
|
| 36 |
```py
|
| 37 |
+
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
| 38 |
|
| 39 |
+
tokenizer = AutoTokenizer.from_pretrained("sagard21/python-code-explainer")
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
+
model = AutoModelForSeq2SeqLM.from_pretrained("sagard21/python-code-explainer")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
```
|
| 44 |
|
| 45 |
## Validation Metrics
|