A Partially Observable Markov Decision Processes (POMDP) framework is used to create an Electrohydrodynamic (EHD) printing closed-loop control system that is capable of making decisions under uncertainty. The POMDP leverages reinforcement learning to obtain the best decisions when interacting with the environment. It has been used in robotics applications and has the potential to automate additive manufacturing technologies, such as EHD printing. This framework facilitates receiving observational feedback from an image processing monitoring system that analyzes high-speed camera images and then finds an optimal action to be executed accordingly. The printed line width forms the hidden state space that is analyzed using image processing techniques along with unsupervised learning algorithms to obtain the transition model. The algorithms were used to cluster the EHD experimental data output and specify the line width before and after an action calculated in pixels using image processing. These algorithms include K-means, Gaussian mixture models (GMM), and Agglomerative clustering. Then, Agglomerative clustering was selected based on a weighted 50% selection criteria between its Silhouette Score and computational time. Noting that the ratio between actual width in micrometer and width in pixels was captured through a polynomial regression model. This state space is partially observed and linked to the recorded highspeed images, which contain a piece of valuable information that is utilized to optimize the ongoing operation. The linking happened through the observational model. As the difference between the first and last groups of the region of interest (ROI) grayscale profile of 200 images received from the stream images represents the observational space of the POMDP. The observed space is discretized to develop a finite set of observations that structure the observational model. This model relies on the observational probabilities computed offline and links each observation after executing an action with its new line width state. The paper aims to establish a POMDP framework that can be used to control EHD printing parameters and operational settings, including plotting speed and standoff distance in real-time. It applies different solution methods and compares their overall performance. The comparison evaluated the computational time, rewards obtained, and convergence to the goal state presented by the desired range of line width. The solution methods covered exact solution approaches as well as approximate solution approaches. Results show a potential application of both approaches in EHD printing settings, as value iteration works well with small POMDP settings and successfully finds an exact optimal solution that declares the action needed to achieve a desired line width state. On the other hand, the point-based value iteration method shows the best approximation solution that can be used in expanded POMDP EHD settings with moderate complexity compared to heuristic search value iterations.