Over the past few years, technological advancements have significantly transformed the manufacturing industry. In this digital era, the shift towards smart manufacturing (SM) is becoming essential for companies to survive in the market. As industries aim to improve efficiency and productivity, there is a growing focus on applying data-driven artificial intelligence (AI) and machine learning (ML) techniques for the purpose of monitoring, prediction, and control of the manufacturing processes. Research in advanced manufacturing has shown that applying these techniques can enhance the effectiveness of manufacturing processes by leveraging data in the decision-making process.Nevertheless, despite the significant advantages offered by AI/ML-based techniques, their adoption in industrial settings has faced several barriers, including limited access to high-quality data, difficulties in scaling models across various settings, and the opacity of "black box" models, which limit trust and acceptance among practitioners, leading to hesitation in adopting such approaches for critical manufacturing operations. This uncertainty in how the results are generated highlights the need to enhance the explainability and reliability of AI systems. One way of tackling these issues is to integrate large language models (LLMs) that can improve the reasoning of otherwise complex and opaque AI/ML systems.However, there are limitations to implementing LLMs in manufacturing contexts. LLMs are pre-trained on unstructured data, which can cause hallucinations; for instance, they may come up with vague or even inconsistent explanations that do not relate well to the manufacturing context. These failures can greatly affect the performance of the manufacturing processes and may lead to costly consequences such as misdiagnosis which could lead to production downtime and material wastage. These risks must be eliminated in order to achieve the maximum benefit of using LLMs in increasing the efficiency of manufacturing operations.To ensure the reliability of LLMs, this work proposes a Context-Aware Multi-Agent (CAMA) framework for manufacturing systems. The framework uses a dynamically updating Knowledge Graph (KG) that incorporates the knowledge of the operators, real-time data, and historical machine data. The CAMA framework, further, establishes and maintains the causal relationship within the data to make accurate diagnosis and specific action plans, thus, leading to the enhancement of the automation of the manufacturing processes. The KG looks to serve as an integrated knowledge bank that helps in the comprehension and application of data for better decision-making.To show the efficacy of the proposed framework, a multi-agent AI system for a mid-scale Fused Deposition Modeling (FDM) 3D printer has been developed. This system is capable of real-time supervision of the manufacturing operation by means of multi-modal sensor fusion, using camera and accelerometer sensors to improve the accuracy and robustness of the data collected. It generates clear, human-interpretable alarm messages based on real-time data, such as vibration information provided by the accelerometer or visual information obtained from the camera. The system also offers recommendations for the possible control actions that may include changing the extrusion speed, the print temperature or even stopping the process to avoid further damage. Therefore, the CAMA framework not only improves the reliability and transparency of the AI-based systems but also helps to improve the overall manufacturing performance. This work showcases how integrating structured knowledge graphs with LLM-based multi agent system can bridge the gap between AI-driven automation and industry adoption.