Smarter Office Automation System

People say "AI is the new Electricity", we wholeheartedly believe in this Quote. Still we believe that today in this tech driven world where problems like climate change, power outage, blackouts , pollution and poverty exists, it's our responsibility to save the old existing Electricity which is the most important thing for survival. We wish to achieve our goal by using the power of Deep Learning and Artificial Intelligence . The commercial office sectors are said to consume about 30% of world's total electricity but seeing around in offices we can clearly predict that there is lot of power being wasted . The purpose of our project is to use the power of Deep Learning and Artificial Intelligence to solve the problem of Electricity Wastage and provide an efficient way of saving Electricity in offices. This will help in saving money for offices and will also benefit environment as, saving electricity will lower the use of fossil fuels used for its generation. Minimal Uses of fossil fuel will help us in taking a step closer to solving problems of climate change and global warming. Electricity saving/storage can help in providing electricity even in rural areas(problem of electricity in rural part of India is very concerning) and will help in increasing their standard of living and production.

As the whole project is build around saving electricity in offices, we have also developed few other Deep Learning models to increase the productivity and security in offices which has lead us to build a complete Office Automation and Security System. The things which sets apart our office automation system with few others that are available today is that the present automation system still requires an Android/IOS app for getting things done, so this completely rips off the idea of being automated. The present office automation system are also very costly .Our office automation system takes care that the hardware is capable enough of handling the complete task with minimal human interaction/interference thus making the system automated in true sense. Relative to present automation systems our product is cheaper.

The Jetson TX2 board was very much useful and essential because no other embedded system has the power of handling/Deploying different Deep Neural Networks at a great speed and performance. Deep Neural nets are the core technology used in building our office automation product, hence getting a fast output from them was very much essential. The task related with video and image analytics were handled with great ease and speed by the jetson board. The reason we used an embedded system is because computer system requires lots of power as well as they can be easily hacked. The jetson board in its max performance mode managed to give a great result with minimum power requirement and less power consumption is the base of our idea. The security provided by Nvidia for the board relieved us from the problem of it being hacked. Thus the Nvidia Jetson Board helped us to implement Artificial Intelligence at Edge with great success.

The project Implementation and testing was carried in our College's Veermata Jijabai Technological Institute research and Industrial lab known as Centre of Excellence(Coe) lab.

* Technical Aspect of the Jetson TX2 and Convolution Neural Network:

The Core algorithms which run in the heart of our project are Deep Convolutional Neural Network(DCNN). Deep Convolution Neural Network are class of deep,feed-forward artificial neural networks that has been successfully been applied to analyzing visual imagery. A Deep convolutional neural network usually contains more than 15 layers of convolution,relu and maxpool etc. Convolution layer is the core building block of a DCNN . The layer’s parameters consist of set of learnable filters, which have a small receptive field, but extend through the full depth of the input volume. Pooling layer is kind of non-linear downsampling. Relu layers provide a non linearity to the entire deep learning model. Thus given an Input image the entire image is parsed through the DCNN network and then the image is classified. There are lots of computation going on each layers and depending on the size of the image the computations can even get complex. This whole process of computation take lots of amount of computational speed and time. There are Nvidia Graphical processing unit (GPU) like Titan Xp for computational system which can help us train and deploy a DCNN at a very high rate but for embedded applications using such a powerful GPU is not possible. This is the scenario when the Nvidia Jetson TX2 embedded system comes into picture. Though the Jetson TX2 board can’t train a DCNN like a GPU for computers but the deployment speed of DCNN in Jetson TX2 is very fast and very much suitable for embedded applications. The 256 CUDA cores of jetson along with 4 core Arm processor and 2 core denver provide a powerful GPU and jetson was able to deploy different kinds of Deep convolutional neural networks used by us during the project with great ease. The Cudnn library provided along with jetpack provide the GPU acceleration in processing our deep learning code. All our programming was done on Python, Keras and tensorflow. We have built 4 different kinds of Deep Learning models in our project all of which can run in parallel due to the GPU power provided by the board. We also wish in future to change our code from python to C++ so as to take advantage of TensorRT , the library for optimizing different layers of CNN by stacking them together in an efficient way to deploy models even faster.

The Whole Implementation of the project is Described below:

1)Automatic turning on/off of Electrical Appliances in office :

For Automatically turning on/off the Electrical Appliances we have used a wireless camera which is has been setup in such a way that the whole room is visible. The camera is connected via an IP address to the Jetson TX2 board. Jetson TX2 board deploys a You Only Look Once(YOLO) Deep Learning algorithm on the frames obtained from the camera to detect the count, position, absence/presence of employees to control electricity. YOLO is a deep convolutional neural Network model(DCNN) which is the state of the art object detection model which offers high speed with precision. Depending on the whether there are employees located in the room the Jetson TX2 board send a request to the server which is hosted on the jetson TX2 board. We have a node js for our server side and google firebase a realtime database for storing the useful contents of the project. Now when the request hits the server the server triggers the hardware which is always connected to the wifi and always listening for the commands to start the appliances.YOLO along with Jetson TX2 board gave us a very accurate and speedy results in real life scenarios for determining if there are people located in a room. Pretrained weights of YOLO were used for the project.

2)Automatic Attendance of employees using Face Detection :

The traditional system of attendance include using a Fingerprint for attendance but the cost of installing such a machine is very high. We try to develop a cheap, reliable and flexible attendance system with the help of a wireless camera and Jetson TX2 board. The wireless camera is mounted on the office/office-room entrance and it continuously captures frames for a specific time which is the allowed time in the office. The employee just have to look for a second in the camera after which the frame captured will be send to the Jetson TX2 Board. The Jetson TX2 board then will deploy a model known as Facenet. Facenet will help us recognize that employee and his/her attendance will be saved on database which can be easily accessed from the app that we have created which is known as the Office Assistant App. Facenet is the state of the art Deep Learning technique for face recognition. Facenet is a Deep Convolution Neural network based on Inception-Resnet-v1 model. Facenet models are trained on Triplet loss method which makes the model capable of creating an encoding of every face in the form of embeddings. Thus we create a database of the Embeddings of all employees in a office .The Embeddings of the different employees can be then classified using a standard machine learning model(Support vector Classifier in our case) which helps us to recognize face. Before the frame is given to Facenet, the frame is ran through Multi-task Convolutional Neural Network(MTCNN). The MTCNN is responsible for face-detection as well as aligning the faces in proper position. Pre trained weights of both MTCNN and facenet were used. In our project we took the embeddings of the 5 members present in the Project and thus their classification of faces was done. We achieved an great 95.63 percent accuracy on classifying faces of the 5 members(offline testing). This method is so flexible that it can be readily be scaled to classify 100 employees in minimum amount of time and complexity. For person who are not the employee in the office are also be recognized as “Not a employee of the office”.

3) Chatbot :

We believe that a chatbot can help tremendously in the office automation space, save costs and save countless human hours that can be utilized for more productive work. Therefore we introduce our chatbot in office assistant app which is a polite, obedient and hilarious chatbot. He can do the following functions: It can greet users, handle basic conversational context, tell jokes, and is highly customizable for the needs of any organization to perform a wide range of HR operations seamlessly, and with minimal changes. Chatbot has special authentication parameters built-in, that restrict the data access to unauthorized personnel. For example: The managers can check attendance and number of hours clocked-in of any day of any employee by simply typing in the name of the employee, thus saving costs and increasing productivity, also due to the chat-app interface, alerts are sent to the manager on a regular basis, notifying him of the other important statistics of the day like crowd count, crowd density in different locations, number of employees who clocked in late etc. It can automate several low level HR features for the employees like scheduling appointments, checking attendance and checking project status. It also helps the employees learn more about their projects with helpful links. With reference to our lab, at any point in time, we can predict the presence of sir in the lab, by using an artificial neural network fed with the details of his presence for a time period of 2 weeks. The chatbot presented in this project combines traditional rule-based chatbot models with advanced deep learning models. Deep Learning models, as amazing as they are, have trouble doing basic tasks like interpretations of calculation-based questions, or telling novel data that it has not been trained on, like telling a joke for example, or even telling the time or remembering a name or context for extended sessions of time.To overcome these shortcomings, we have integrated a rule-based approach with the deep-learning models, to create a novel architecture.The deep-learning model used in this project is sequence-to-sequence model which consists of two Recurrent Neural Networks(RNNs) - an encoder and a decoder.

4)Heatmaps :

The present automation systems in use fails to take into account one of the most important thing for office employees that is the security in emergency cases. We have developed a Deep Convolutional Neural Network which is capable of producing 2D Heat Maps, which are very useful in analysing the crowd density and count in case of emergency situations like firebreak, theft, terrorist attack and many such cases to act efficiently and quickly. Heatmaps gives the count of people present in room. This can be very useful in cases of crowd management. Apart from the security, employees can use Heatmaps to check the crowd in the canteens, cafes, or other rooms in offices that are usually crowded to plan their visit at these places according to the crowd present thus saving their time, which is otherwise wasted waiting in long queues. Thus any employee can request the heatmap of any room in the office anytime from their Office Assistant app. The request will be sent to the Jetson TX2 which will process the image of the desired location obtained from wireless cameras and produce the heatmap which will be sent to the app of the user who asked for it.

Heatmap and chatbot deployment is shown on the video submitted

5)Resource Management in offices:

There are many equipments and stationery available in offices which are used by employees.Some offices need to keep a track on which equipment is used by which specific employee(eg: In our current working lab we have to maintain a record in a register of the industrial equipments that we use).Offices use traditional ways of "entering" the data in some kind of database.This is an inefficient and time consuming process and can also lead to human errors. So we have created a convolution neural network(CNN) deployed on Jetson TX2 which is trained to classify different equipments.The Employee just have to take an image of the equipment he is going to use through his Office Assistant app after which the image will be sent to the jetson TX2 and will be passed through a CNN. The CNN will classify the image and will maintain the name of the equipment used and the employee who is using it in the database. For our project purpose and also for smooth functioning of our college research lab we have used CNN to classify 5 electrical components namely BeagleBone, raspberry pi, arduino, weemos,hackerafone .

6)Office Assistant App:

The Implementation of Chatbot , Heatmaps and Resource management systems is done on the Android App that we created called as Office Assistant App. It also Contains the attendance record of the employees.

THANK YOU