Photo by Arseny Togulev on Unsplash

Natural Language Processing (NLP)

Transformers Pipeline

🤗 Transformers, why are they so damn cool?

A few years ago, I developed a few NLP models. I have recently noticed that many things have changed positively in NLP world after 2018. This rapid evolution of Transformers has pioneered powerful tools for natural language processing. Now it is possible to do a lot with three lines of code. If you’re wondering how it is possible, the answer is hidden in the Transformers. You know they are everywhere :)

The original Transformer architecture looked like this below picture, not like the above:), with the encoder on the left and the decoder on the right:

Don’t worry, I’m not going to make detailed and fancy explanations about this picture. Those who are interested can access the explanations from this link. Let’s see what it can do instead.

The Hugging Face Transformers provides thousands of pre-trained models to perform tasks on texts such as classification, information extraction, question answering, summarization, translation, text generation, and more in over 100 languages. Its aim is to make cutting-edge NLP easier to use for everyone. The Model Hub contains thousands of pre-trained models that anyone can download and use. It also contains a large number of datasets for NLP projects.

In the model hub, many NLP tasks have a pre-trained pipeline ready to go. If we put our words into action, for example, we can easily classify positive versus negative texts.

from transformers import pipeline
# Allocate a pipeline for sentiment-analysis
classifier = pipeline('sentiment-analysis')
classifier('We are very happy to introduce pipeline to the transformers repository.')
[{'label': 'POSITIVE', 'score': 0.9996980428695679}]

First, import the pipeline and it cares the rest as you see above examples. (Here the second line of code downloads and caches the pre-trained model used by the pipeline, while the third evaluates it on the given text.) The result is “positive” with a confidence of 99.97%, awesome! isn’t it?

Some of the currently available pipelines that you can easily use are:

1. feature-extraction (get the vector representation of a text)
2. fill-mask
3. ner (named entity recognition)
4. question-answering
5. sentiment-analysis
6. summarization
7. text-generation
8. translation
9. zero-shot-classification

Let’s have a look at one more example that I find very useful and time-saving. Summarization:

from transformers import pipeline
summarizer = pipeline("summarization")
summarizer("""
America has changed dramatically during recent years. Not only has the number of graduates in traditional engineering disciplines such as mechanical, civil,electrical, chemical, and aeronautical engineering declined, but in most of the premier American universities engineering curricula now concentrate on and encourage largely the study of engineering science. As a result, there are declining offerings in engineering subjects dealing with infrastructure, the environment, and related issues, and greater concentration on high technology subjects, largely supporting increasingly complex scientific developments. While the latter is important, it should not be at the expense of more traditional engineering. 
Rapidly developing economies such as China and India, as well as other industrial countries in Europe and Asia, continue to encourage and advance the teaching of engineering. Both China and India, respectively, graduate six and eight times as many traditional engineers as does the United States. Other industrial countries at minimum maintain their output, while America suffers an increasingly serious decline in the number of engineering graduates and a lack of well-educated engineers. 
""")

The output is also great;

[{'summary_text': 'America has changed dramatically during recent years . The number of engineering graduates in the U.S. has declined in traditional engineering disciplines such as mechanical, civil, electrical, chemical, and aeronautical engineering . Rapidly developing economies such as China and India, as well as other industrial countries in Europe and Asia, continue to encourage and advance engineering .'}]

As you can see, it is like a plug-and-play code that you can easily use in your projects. You have many reasons, if you ask, why should I use a transformer? here are some of those;

1. Easy-to-use state-of-the-art models,
2. Lower compute costs, smaller carbon footprint,
3. Choose the right framework for every part of a model’s lifetime,
4. Easily customize a model or an example to your needs,

If you ask, why shouldn’t I use transformers?

Huggingface transformers currently offer free democratic use as an open-source model with API support, my big concern there is no strong guarantee that this will always be the case. l hope that they always support the open-source community and its development.

There are many more features you will discover about 🤗 Transformers. As you can see, with these transformers, you can create great works in the NLP world in 10 minutes. I would recommend that you use your own models with transformers without losing any time.

Sources:

GitHub - huggingface/transformers: 🤗 Transformers: State-of-the-art Natural Language Processing…

Understanding Transformers, the Data Science Way - KDnuggets