Awesome C++
Section: Machine Learning · A high-performance neural network inference computing framework optimized for mobile platforms. [BSD]
FreshScore 94
Entry
ncnn
Appears in 6 awesome lists
High-performance neural network inference framework optimized for mobile platforms. No third-party dependencies, cross-platform, and runs faster than all known open-source frameworks on mobile CPU. Powers Tencent apps including QQ, WeChat, and Pitu. BSD-3-Clause licensed.
Open github.comtencent/ncnn
Found in these lists
Awesome LLMOps
Section: Optimizations · ncnn is a high-performance neural network inference framework optimized for the mobile platform.
ActiveScore 75
Awesome Model Quantization
Section: Inference and hardware · Mobile neural network inference, including INT8 deployment.
FreshScore 83
Awesome Open Source AI
Section: 11. Specialized Domains · High-performance neural network inference framework optimized for mobile platforms. No third-party dependencies, cross-platform, and runs faster than all known open-source frameworks on mobile CPU. Powers Tencent apps including QQ, WeChat, and Pitu. BSD-3-Clause licensed.
FreshScore 89
Awesome Vulkan
Section: Libraries · High-performance neural network inference framework with Vulkan based GPU inference. [BSD 3-clause]
ActiveScore 68
awesome-cpp
Section: Other · ncnn is a high-performance neural network inference framework optimized for the mobile platform
FreshScore 79
Often listed next to it
m2cgen
Transpile trained ML models into other languages. sklearn-porter - Transpile trained scikit-learn estimators to C, Java, JavaScript and others. mlflow - Manage the machine learning lifecycle, including experimentation, reproducibility and deployment. skll - Command-line utilities to make it easier…
In 16 listsDetails
XGBoost
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Flink and DataFlow. [Apache2]
In 11 listsDetails
vLLM
State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.
In 11 listsDetails
CatBoost
General purpose gradient boosting on decision trees library with categorical features support out of the box. It is easy to install, contains fast inference implementation and supports CPU and GPU (even multi-GPU) computation.
In 10 listsDetails
Caffe
is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR)/The Berkeley Vision and Learning Center (BVLC) and community contributors.
In 10 listsDetails
llama.cpp
Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.
In 10 listsDetails
FAISS
is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning. Faiss is written in C++ with complete…
In 8 listsDetails