3 years ago

Category: Jenkins, github, automation, shell, machine-learning

Here i am going to share a project on automated system i build in the training of MLOPS under the mentorship of Vimal Daga sir (LinuxWorld Informatics pvt. ltd).. Overview : Whenever we go for machine learning or deep learning using neural network we have to create some layered architecture and to get the better performance we have to modify and check the model architecture that for which particular model architecture given dataset, after the learning provides a model that is giving the accuracy in prediction, more than a certain value. But this is a time consuming process because in a deep learning architecture to train a model we can have a large dataset, so to every time train a model and check accuracy and modify architecture and again start learning and simultaneously monitor that if the process is not been failed because of any reason.

We have used “sed” shell command to add everytime two layers after pooling layer in the model training code, one is convolution layer with “relu” function and next pooling layer, so after model tweaking we will have two more consecutive (convolution+pooling) layers then fully connected layer, and number of epochs will be double every time when we will tweak the model. After the commands to change model architecture we will run exit 1 exit 1 indicate that the job has been failed so we will trigger job4, this is because there is going to be a loop in job3 and job4, so each time when accuracy is not reach to the required accuracy job3 will fail and this fail will trigger to job4, and when the accuracy will be greater than or equal to 0.99 the job will be successfull so job3 will not trigger job4 and execution will be terminated and we will get the model with required accuracy.