Phi-2 Integrations: Unlocking High-Performance NLP with OpenVINO and Spark NLP

The landscape of natural language processing (NLP) is advancing at a rapid pace, with new models and technologies constantly pushing boundaries. The latest addition, Phi-2, takes center stage as a powerful Transformer model that balances efficiency, performance, and openness. Developed with 2.7 billion parameters, Phi-2 integrates seamlessly with OpenVINO, enabling quantization on CPUs for improved performance. But Phi-2 isn’t just about raw power — it’s also designed for accessibility and usability, encouraging responsible AI research by addressing key concerns like toxicity and bias.

In this article, we explore Phi-2’s innovative features, its integration with OpenVINO, and how it empowers developers and researchers alike through Spark NLP.

Introducing Phi-2: Small But Mighty

Table 1. Averaged performance on grouped benchmarks compared to popular open-source SLMs.

Table 2. Comparison between Phi-2 and Gemini Nano 2 Model on Gemini’s reported benchmarks.

Phi-2 is a powerful Transformer model designed to tackle NLP tasks that require common sense reasoning, language understanding, and logical deduction. Trained on the same sources as its predecessor, Phi-1.5, Phi-2 includes additional curated synthetic texts and filtered websites for enhanced safety and educational relevance. This open-source model, which performs comparably to much larger models, offers significant flexibility for developers while requiring fewer resources.

Despite its relatively small size in terms of parameters, Phi-2 is no lightweight in performance. Benchmark tests reveal that Phi-2 performs nearly at state-of-the-art levels among models with fewer than 13 billion parameters. And unlike some larger models, Phi-2 is designed without reinforcement learning from human feedback (RLHF). This makes it a particularly valuable tool for researchers exploring critical areas in NLP like bias mitigation, controllability, and toxicity reduction, all while providing transparency and openness.

Seamless Integration with OpenVINO for Optimized Performance

One of the standout features of Phi-2 is its integration with OpenVINO, Intel’s toolkit that enables AI model deployment on a range of hardware with performance optimizations. Through OpenVINO, Phi-2 achieves efficient quantization for INT4 and INT8 on CPUs. This allows developers to deploy Phi-2 for real-time applications with optimized performance, preserving accuracy while significantly improving speed.

This level of integration means that Phi-2 not only performs well on smaller devices but also requires less computational power overall, making it suitable for a wide range of applications. Imagine using Phi-2 in applications like customer support chatbots, virtual assistants, or educational content generators — all areas where speed, accuracy, and reliability are essential.

Phi-2’s integration with OpenVINO is also designed to be straightforward and scalable, ensuring that users, even those with limited machine learning experience, can leverage these optimizations. The model is easy to adopt and integrate into existing systems, thanks to a clear and accessible API, saving developers time and effort.

Getting Started with Phi-2 in Spark NLP

One of the primary ways to deploy and interact with Phi-2 is through Spark NLP. Spark NLP, a popular library for scalable natural language processing in big data environments, allows developers to set up and run Phi-2 with just a few lines of code. Here’s a quick look at how easy it is to get Phi-2 up and running.

First, we use the DocumentAssembler component in Spark NLP to prepare the text data:

doc_assembler = DocumentAssembler() \
      .setInputCol("text") \
      .setOutputCol("document")

Then, we initialize [Phi2Transformer](https://sparknlp.org/models?annotator=Phi2Transformer)with customizable parameters. Here’s an example pipeline setup:

phi2 = Phi2Transformer \
        .pretrained() \
        .setInputCols(["document"]) \
        .setMaxOutputLength(50) \
        .setOutputCol("phi2_generation")

With this setup, developers can easily tailor Phi-2’s performance to suit specific needs. The model supports adjustable parameters such as maxOutputLength, doSample, temperature, and more, allowing users to control output length, sampling behavior, and even probability filtering.

For example, if you’re building a chatbot, you might set doSample=True to introduce more variability in responses, or use topK and topP filtering to fine-tune response quality.

Practical Applications and Benefits of Phi-2

Phi-2’s adaptability and ease of integration make it ideal for a variety of real-world applications:

• Customer Support: With Phi-2’s ability to understand and respond to user queries in natural language, it can power chatbots and virtual assistants to handle customer inquiries quickly and accurately.
• Education and E-Learning: Phi-2’s general knowledge and language understanding make it suitable for creating interactive learning tools that can provide explanations, answer questions, and even offer tutoring in specific subjects.
• Content Generation: In industries where generating text-based content is a priority, Phi-2’s efficiency and configurability make it a valuable tool for crafting personalized responses, summaries, and insights.

Beyond these applications, Phi-2’s open-source nature also makes it a valuable research tool. Researchers can use Phi-2 to explore ways of mitigating bias, reducing toxicity, and improving model safety, contributing to the responsible development of NLP technologies.

Phi-2 in Action: Example Pipeline and Output

To showcase Phi-2’s capabilities, let’s look at a simple example where it generates a response from a user’s input text:

import sparknlp
from sparknlp.base import *
from sparknlp.annotator import *
from pyspark.ml import Pipeline

documentAssembler = DocumentAssembler() \
    .setInputCol("text") \
    .setOutputCol("documents")

phi2 = Phi2Transformer.pretrained("phi2") \
    .setInputCols(["documents"]) \
    .setMaxOutputLength(50) \
    .setOutputCol("generation")

pipeline = Pipeline().setStages([documentAssembler, phi2])
data = spark.createDataFrame([["My name is Leonardo."]]).toDF("text")
result = pipeline.fit(data).transform(data)
result.select("generation").show(truncate=False)

This code produces text responses based on the input, highlighting Phi-2’s language generation and understanding abilities. For example, given input about a person’s interests, Phi-2 might generate a meaningful response that adds context or elaborates on the topic.

Looking Ahead: The Future of Phi Models

Phi-2 represents a shift toward more accessible and open NLP models, but it’s just the beginning. The development of Phi models highlights an industry trend: creating efficient, ethical, and adaptable NLP solutions. By building on Phi-2’s framework, future models can continue to evolve, benefiting from community contributions to tackle ongoing challenges in AI ethics, safety, and scalability.

As the Phi series continues to grow, it opens new avenues for the research community and developers to build upon and refine, ensuring that AI technology not only advances but does so responsibly.

Conclusion

Phi-2 is more than just a model; it’s a practical tool that marries high performance with responsible design. Its integration with OpenVINO makes it efficient and versatile, while Spark NLP allows easy deployment and customization. As Phi-2 and future iterations continue to unfold, they promise to drive innovation and ethical development in the NLP landscape.

With Phi-2, developers, researchers, and the broader AI community have a new, accessible way to explore the potential of language models — bringing AI a step closer to real-world usability and societal benefit.

📄 Spark NLP 5.4.0 Release Notes

For further information and helpful resources, explore these links:

• GitHub — Spark NLP Repository: Access the codebase and contribute.
• Models Hub — Explore Models: Discover and download pre-trained models.
• More Examples — Spark NLP Workshop: tutorials and hands-on examples.

❤️ Community and Support

• **Slack:* Join the Spark NLP community for live discussions with the team and other users.*
• **Discussions*: Connect with the community, share insights, and showcase your Spark NLP projects.*
• **Medium*: Read official Spark NLP articles on Medium.*
• YouTube: Watch video tutorials on the Spark NLP YouTube channel for in-depth learning.