3697e899f6- Track per-model and per-turn token usage in MLChatAgentRunner and MLPlanExecuteAndReflectAgentRunner using AgentTokenTracker - Extract token counts from LLM responses via FunctionCalling interface - Add include_token_usage opt-in flag for agent responses - Emit token usage via CustomEvent for AGUI streaming agents - Add utility methods in AgentUtils for model metadata resolution, token tensor creation, and structured per-model usage logging - Suppress sub-agent token logging to prevent double-logging in PER - Fix NPE for old-style AGUI agents with conversational memory - Add comprehensive unit tests across all components Signed-off-by: Rithin Pullela <rithinp@amazon.com> Signed-off-by: rithin-pullela-aws <rithinp@amazon.com> |
||
|---|---|---|
| .github | ||
| build-tools | ||
| client | ||
| common | ||
| docs | ||
| gradle/wrapper | ||
| memory | ||
| ml-algorithms | ||
| plugin | ||
| release-notes | ||
| scripts | ||
| search-processors | ||
| spi | ||
| .eclipseformat.xml | ||
| .gitignore | ||
| .whitesource | ||
| build.gradle | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| DEVELOPER_GUIDE.md | ||
| gradlew | ||
| gradlew.bat | ||
| LICENSE | ||
| lombok.config | ||
| MAINTAINERS.md | ||
| NOTICE.txt | ||
| README.md | ||
| settings.gradle | ||
| TRIAGING.md | ||
OpenSearch Machine Learning Commons
Machine Learning Commons for OpenSearch is a new solution that make it easy to develop new machine learning feature. It allows engineers to leverage existing opensource machine learning algorithms and reduce the efforts to build any new machine learning feature. It also removes the necessity from engineers to manage the machine learning tasks which will help to speed the feature developing process.
Problem Statement
Until today, the challenge is significant to build a new machine learning feature inside OpenSearch. The reasons include:
- Disruption to OpenSearch Core features. Machine learning is very computationally intensive. But currently there is no way to add dedicated computation resources in OpenSearch for machine learning jobs, hence these jobs have to share same resources with Core features, such as: indexing and searching. That might cause the latency increasing on search request, and cause circuit breaker exception on memory usage. To address this, we have to carefully distribute models and limit the data size to run the AD job. When more and more ML features are added into OpenSearch, it will become much harder to manage.
- Lack of support for machine learning algorithms. Customers need more algorithms within Opensearch, otherwise the data need be exported to outside of OpenSearch, such as s3 first to do the job, which will bring extra cost and latency.
- Lack of resource management mechanism between multiple machine learning jobs. It's hard to coordinate the resources between multi features.
In the meanwhile, we observe more and more machine learning features required to be supported in OpenSearch to power end users’ business needs. For instance:
- Forecasting: Forecasting is very popular in time series data analysis. Although the past data isn’t always an indicator for the future, it’s still very powerful tool used in some use cases, such as capacity planning to scale up/down the service hosts in IT operation.
- Root Cause Analysis in DevOps: Today some customers use OpenSearch for IT operations. It becomes more and more complicated to identify the root cause of an outage or incident since it needs to gather all the information in the ecosystem, such as log, traces, metrics. Machine learning technique is a great fit to address this issue by building topology models of the system automatically, and understanding the similarity and casual relations between events, etc.
- Machine Learning in SIEM: SIEM(Security Information and Event Management) is another domain in OpenSearch. Machine learning is also very useful in SIEM to help facilitate security analytics, and it can reduce the effort on sophisticated tasks, enable real time threat analysis and uncover anomalies.
Solution
The solution is to introduce a new Machine Learning library inside the OpenSearch cluster. The major functionalities in this solution include:
- Unified Client Interfaces: clients can use common interfaces for training and inference tasks, and then follow the algorithm interface to give right input parameters, such as input data, hyperparameters. A client library will be built for easy use.
- ML Plugin: ML plugin will help to initiate the ML nodes, and choose the right nodes and allocate the resources for each request, and manage machine learning tasks with monitoring and failure handing supports, and store the model results; it will be the bridge for the communication between OpenSearch process and ML engine.
- ML Engine: This engine will be the host for ML algorithms. Java based machine learning algorithms will be supported in the first release.
This solution makes it easy to develop new machine learning features. It allows engineers to leverage existing open-source machine learning algorithms, and reduce the efforts to build any new machine learning feature. It also removes the necessity from engineers to manage the machine learning tasks which will help to speed up the feature developing process.
How to use it for new feature development
Contributing
See developer guide and how to contribute to this project.
Code of Conduct
This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ, or contact opensource-codeofconduct@amazon.com with any additional questions or comments.
Security
If you discover a potential security issue in this project we ask that you notify OpenSearch Security directly via email to security@opensearch.org. Please do not create a public GitHub issue.
License
This project is licensed under the Apache v2.0 License.
Copyright
Copyright 2020-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.