As a part of Make clear, AWS affords enterprises a function, dubbed FMEval, which is an open-source LLM analysis library to assist knowledge scientists and ML engineers consider LLMs earlier than deciding to make use of it for a selected use case.
“FMEval offers the power to carry out evaluations for each LLM mannequin endpoints or the endpoint for a generative AI service as an entire. FMEval helps in measuring analysis dimensions comparable to accuracy, robustness, bias, toxicity, and factual data for any LLM,” the cloud service supplier wrote in a weblog put up.
Enterprises can use EMEval to guage LLMs hosted on both AWS or third-party platforms, comparable to ChatGPT, HuggingFace, and LangChain, it added.
