How NPU still be useless for third party’s developers because of fragmented API with complicated installations and turns as gimmick only when using built-in features

When NPU is going to become a part of main component in the computer, developers still no interested to involve because of complicated setup. Meanwhile, the bundled features still perform notoriously worse than user’s expectations that the entire eco system of PC industries doubt readiness of rolling NPU into commercial market.

Photo by Aerps.com on Unsplash

I totally halted for publishing new Medium articles in 4.5 years ago as that time I was preparing to apply bachelor degree of computer and data engineering (CDE) into an university with senior entry, but ultimately the authority decided to put me back to year one again, which generically called “electrical engineering” even though I have higher diploma of software engineering certification already. Worse still, my year one’s credits was ineligible for returning CDE with high confidence that I decide to apply information engineering (INFE) as applied major now. However, when ChatGPT released to general at late 2022, every industries have been started parsing tasks to AI that its maximising efficiency of productivity with fewer human resources. Currently, my selective courses in final year was dominated by AI related objects already that I am experiencing learning duplicated subjects from two enrolled courses in this semester. Therefore, the debut market release of AI led to a large invest of related industries that people from enterprises to academies.

Generally, for individual, it is preferred to proceed it locally instead of remote service from the Internet due to security reason. Hence, modern enterprise-oriented computers often ship neural processing unit (NPU) as a selling point, even Microsoft has a classification named as “Copilot+ PCs” for any Windows pre-installed computers with NPU provided with satisfied computations. Hilariously, for typical users, only selective Windows’ built-in features can use NPU resources and there are only limited numbers of software provides AI features that can be run in local device. Despite that, the software often uses OpenVINO — a SDK of Intel NPU such that it completely shut the door for those PC using AMD CPU. Although AMD has it owns NPU SDK called Ryzen AI, most program does not implement it as its SDK is simply a miniforge’s environment with vulnerability of breaking dependencies when attempting to install or update packages, which the installer ships legacy packages when creating miniforge environments. Result of “pip freeze” in the Ryzen AI environment is exported as below:

absl-py==2.4.0
aianalyzer @ file:///C:/SDKs/RyzenAI/1.7.1/aianalyzer-1.7.1-py3-none-any.whl#sha256=7d1417162dab886fca7a16bbf20aabc6c67f2c697863f5a00c8f1ccecfc5e816
aiohappyeyeballs==2.6.1
aiohttp==3.13.5
aiosignal==1.4.0
amd-quark @ file:///C:/SDKs/RyzenAI/1.7.1/amd_quark-0.11rc1-py3-none-any.whl#sha256=99c12fc866c476cb31514b2bafb5f3ba50a0432cc78f43869908a4405fd3cc3c
amd_atom @ file:///C:/SDKs/RyzenAI/1.7.1/amd_atom-1.7.1-cp312-cp312-win_amd64.whl#sha256=c3166b9c66d24c2f04fa257647f6bd6c983ee9689866cc318ee80e24b0135268
annotated-types==0.7.0
anyio==4.13.0
attrs==26.1.0
beartype==0.22.9
blinker==1.9.0
cachetools==7.0.5
certifi==2026.2.25
charset-normalizer==3.4.7
click==8.3.2
colorama==0.4.6
coloredlogs==15.0.1
colorlog==6.10.1
contourpy==1.3.3
cycler==0.12.1
dacite==1.9.2
dataclass-wizard==0.36.2
datasets==4.8.4
device-essentials-strx-overlay @ file:///C:/SDKs/RyzenAI/1.7.1/device_essentials_strx_overlay-1.7.1-py3-none-any.whl#sha256=5828fa2f7879dc548155f3ace16eb4578c075d3c672b566f79d03f1584f5ee23
diffusers==0.36.0
dill==0.4.1
dnspython==2.8.0
dotenv==0.9.9
et_xmlfile==2.0.0
evaluate==0.4.6
filelock==3.25.2
Flask==3.1.3
flatbuffers==25.12.19
flexml-lite @ file:///C:/SDKs/RyzenAI/1.7.1/flexml_lite-1.7.1-py312-none-win_amd64.whl#sha256=2334bad4bf7a6e2a2d835cc79aa52c62a431259e974cc975389fa9f8ee0696dc
flexmlrt @ file:///C:/SDKs/RyzenAI/1.7.1/flexmlrt-1.7.1-py312-none-win_amd64.whl#sha256=ff0a17faef6b3cf851ee6dfa176bc284730ba9c1386124a610b039540eb9d34d
fonttools==4.62.1
frozenlist==1.8.0
fsspec==2026.2.0
graphviz==0.21
greenlet==3.1.1
h11==0.16.0
hiredis==3.3.1
httpcore==1.0.9
httpx==0.28.1
huggingface_hub==0.36.2
humanfriendly==10.0
idna==3.11
ImageIO==2.37.3
immutabledict==4.3.1
importlib_metadata==9.0.0
iniconfig==2.3.0
itsdangerous==2.2.0
Jinja2==3.1.6
joblib==1.5.3
json5==0.14.0
kiwisolver==1.5.0
lazy-loader==0.5
lightning-utilities==0.15.3
llvm-aie-lightweight @ file:///C:/SDKs/RyzenAI/1.7.1/llvm_aie_lightweight-1.7.1-py3-none-win_amd64.whl#sha256=0afa2bb4983baffdd451744f86579c70ce5ae68af8d0804dd25147d908cc5b4a
markdown-it-py==4.0.0
MarkupSafe==3.0.3
matplotlib==3.10.8
mdurl==0.1.2
ml_dtypes==0.5.4
model-generate @ file:///C:/SDKs/RyzenAI/1.7.1/model_generate-1.7.1-py3-none-any.whl#sha256=4099f64a3a5f75fb03616b9153a113d5b0d7b233d881119ce6acb99d338ce03d
mpmath==1.3.0
multidict==6.7.1
multiprocess==0.70.19
narwhals==2.19.0
networkx==3.6.1
ninja==1.13.0
numpy==1.26.4
onnx==1.18.0
onnx-ir==0.2.0
onnx-tool==0.9.0
onnxoptimizer @ file:///C:/SDKs/RyzenAI/1.7.1/onnxoptimizer-0.3.19-cp312-cp312-win_amd64.whl#sha256=e6f740f9ac54a292f4b16cadb337506899d99319f5a6ad9df43efd9dc90c1559
onnxruntime-genai-directml-ryzenai @ file:///C:/SDKs/RyzenAI/1.7.1/onnxruntime_genai_directml_ryzenai-0.11.2-cp312-cp312-win_amd64.whl#sha256=7ac886edd77d94d666fd86b5f79667b360171ee86da4b9742d82e3a1c3beb6cb
onnxruntime-vitisai @ file:///C:/SDKs/RyzenAI/1.7.1/onnxruntime_vitisai-1.23.3-cp312-cp312-win_amd64.whl#sha256=ecd7894242ba52ab7a92d02e2622db7654f1b23fd66b7e3cd6e77d8564245cf6
onnxruntime_extensions==0.15.0
onnxruntime_providers_ryzenai @ file:///C:/SDKs/RyzenAI/1.7.1/onnxruntime_providers_ryzenai-0.11.1-py3-none-win_amd64.whl#sha256=7fe271a3a561f1216cbf2142e9a78dbbd0315fdd7a3e177ce475cf63ddab3f17
onnxscript==0.5.7
onnxsim @ file:///C:/SDKs/RyzenAI/1.7.1/onnxsim-0.4.36-cp312-cp312-win_amd64.whl#sha256=86bd270a55de14d9cf69b436f531f0e443fcbb9b6c5c2126a748fe5ecb37aa07
onnxslim==0.1.90
opencv-python==4.11.0.86
openpyxl==3.1.5
ortools==9.14.6206
packaging==26.0
pandas==2.3.3
pillow==12.2.0
playwright==1.14.1
plotly==6.7.0
pluggy==1.6.0
propcache==0.4.1
protobuf==6.31.1
psutil==7.2.2
pyarrow==23.0.1
pydantic==2.12.5
pydantic_core==2.41.5
pyee==8.2.2
Pygments==2.20.0
pymongo==4.16.0
pyparsing==3.3.2
pyreadline3==3.5.4
pytest==9.0.3
python-dateutil==2.9.0.post0
python-dotenv==1.2.2
pytz==2026.1.post1
PyYAML==6.0.3
redis==7.4.0
regex==2026.4.4
requests==2.33.1
rich==14.3.4
ryzen-ai-lt @ file:///C:/SDKs/RyzenAI/1.7.1/ryzen_ai_lt-1.7.1-py3-none-any.whl#sha256=3ed04b8c2a90acde8f9726af09b72f9a37b51fa38c03af4141d57768d24d16d7
ryzenai-dynamic-dispatch @ file:///C:/SDKs/RyzenAI/1.7.1/ryzenai_dynamic_dispatch-1.7.1-cp312-cp312-win_amd64.whl#sha256=4751e47b7634ab197ab7169ac66b4373d700f9da3b874c6f498e02e00be54ec6
ryzenai_onnx_utils @ file:///C:/SDKs/RyzenAI/1.7.1/ryzenai_onnx_utils-1.7.1-py3-none-any.whl#sha256=710d1a78cfb065143a451a93f01665e2c12d528a390e1ee3b479111b3ce057eb
safetensors==0.7.0
scikit-image==0.25.2
scipy==1.17.1
sentencepiece==0.2.1
setuptools==82.0.1
six==1.17.0
sympy==1.14.0
tabulate==0.10.0
tifffile==2026.3.3
tokenizers==0.22.2
torch==2.4.1
torchmetrics==1.4.1
tqdm==4.67.3
transformers==4.57.6
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2026.1
urllib3==2.6.3
vaie-cpplus @ file:///C:/SDKs/RyzenAI/1.7.1/vaie_cpplus-1.7.1-py3-none-win_amd64.whl#sha256=c234adf8a5e257108c3588b1293478f9856f5861c36dff2fdbdbc2bd20381140
vaie-overlay @ file:///C:/SDKs/RyzenAI/1.7.1/vaie_overlay-1.7.1-py3-none-win_amd64.whl#sha256=e8e001eaafe75b70e87e48ec8afe9c66129eb85010dadf135aa116c8a3a374bf
vaitrace @ file:///C:/SDKs/RyzenAI/1.7.1/vaitrace-1.7.1-py3-none-any.whl#sha256=2b9cc0b8c1700701d0b15832644655bc1d8794214b4cf851289b284a654c2ad4
voe @ file:///C:/SDKs/RyzenAI/1.7.1/voe-1.7.1-py3-none-win_amd64.whl#sha256=8c5643ca96201b135e27c002df8049a2c3d0edb9393b7f75d11925bd4cd9dc90
waic @ file:///C:/SDKs/RyzenAI/1.7.1/waic-1.7.1-py3-none-any.whl#sha256=d292c956d273e0eaad2aac7d7f5781d276b00f939fb53995d2597d8a5ce3ce90
waitress==3.0.2
websockets==10.4
Werkzeug==3.1.8
wheel==0.43.0
xxhash==3.6.0
yarl==1.23.0
zipp==3.23.0
zstandard==0.25.0

As you can see, especially for those proprietary or optimised packages only can be installed by mapping Ryzen AI install location only and it cannot be reproduced in another Python virtual environments as the tailor-made environment also bundled and utilize necessaries driver at the same time. Therefore, with no feasibility and flexibility of features expansion, the developers no longer to be interested to invest AI model deployment in AMD platform or elevating difficulty by purely using C/C++ to write code rather than sticking Python even though it has mature cycle of AI packages with entry level of coding.

Then talk back to built-in features, Microsoft states three main features utilize NPU resources:

• Image processing (in both creation and modification)
• Live caption with real-time translation
• Intelligence searching

In my case, the latter two features are often used, but their performance is generally unsatisfied, especially when handling complex cases that the AI tend to yield inaccurate result in practise. In images enhance search, the AI seem to have strong searching performance in simplified, short prompts applied, but it still cannot extract proper results when non-common words applied in prompt. For example, prompting “meme” can provides all images contain viral content, no matter is newly created or not. When proprietary term is applied (“Flutter desktop” for instance), the search refuses to yield related screenshot of my Flutter example program of my recently released package. Meanwhile, the same image was indexed if “Material UI” is prompted. This is because unique spot of the screenshot is too small that AI may ignore small details to classify as Flutter desktop (obviously refer to the window). Combined the test result, it can be concluded that the AI model can be searched in large scope only with worse performance if precise result is demanded.

The enhanced search can retrieve pictures with corresponded simple keyword where store locally, especially “mpcas_ummr.png” was created by me, which using Spider Man wearing glass meme template and have been posted it into X already

A counter example of enhance search that the opened image does not listed when prompting “Flutter desktop” even though it is. However, the same image was indexed if “Material UI” (design language for the Flutter program) is prompted instead

On the other hand, live captions with instant translation is another AI features I frequently uses that it becomes handy to acknowledge content of foreign language. First time I use it for an online live concert that serval interactive were occurred to make more immersive experience through participation. The problem is the caption cannot properly fetched when instrument was played simultaneously. As a result, I felt unaligned with other audience that I cannot acknowledge what the performer shouted out for what. Another use case of live translation is watching foreign TV series in original spoken language with absence of subtitle. This is most likely happened in streaming service from local TV channel website as a strategy of cost cutting of the operation (but it has been dubbed already). In practice, the caption was unable to synchronize content that some quotes may never be retrieved. This definitely is a severe pain point for audience who not familiar with the foreign language that they may lose interest when they cannot interpret sufficient content without assistance tool operate properly. Hence, it cannot assist audience to acknowledge content in foreign language instantly owing to poor performance of live translation.

In summary, with unsatisfied performance of built-in AI features, it has high demand of developing new softwares, which capable to use NPU resource that they can deliver new local AI services or overcome constraints of existed features. However, fragmentation of toolkit causes the complexity of development that the developers tend to adopt in one platform with maximum flexibility of expansion. AMD failed to fulfil demands from the developers that most software deployed AI features through the solution provided from the competitor. With insufficient variety, the benefits of NPU cannot be utilize properly for end-user scope.