Final

README.md

@@ -0,0 +1,40 @@
+# Video Based Dynamic Human Authentication System For Access Control
+
+## Working of our Solution
+
+Though there exists a technology for face recognition based authentication, dynamic human recognition based authentication is highly challenging. For a given entrance gate a hardware-software solution is needed to identify every unique person who enters or exit the gate, with log of all previous entry/exit time, photo/videos recorded. 
+- At the time of entry the faces of people are captured via the security cameras.
+-  The images of the people entering are checked whether they are registered to enter the premises or not.
+     -  if they are registered they can easily enter the premises. 
+     - Otherwise the individual is not allowed to enter the premises. That means there is no previous history of an individual. 
+        - The system should immediately alert the security if it is a new person and the security will decide to allow/restrict that person entering inside the premises. 
+            -  The person is allowed to enter the premises only if he/she  is registered as a guest or as a daily comer. 
+- Security can check the whole log maintained anytime he  wishes to from his dashboard.
+
+Our system  learns from its previous history of videos/images dynamically to allow a known person. For a given size of the gate, the number of cameras with optimal resolution required is also to be worked out as part of solution.
+
+
+## UML Diagram 
+
+### ENTRY
+
+[![](https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG5QZXJzb24gLT4-IENhbWVyYTogRmV0Y2hlcyBldmVyeSBmcmFtZVxuQ2FtZXJhIC0tPiBBV1MgYW5kIERhdGFiYXNlOiBQcm9jZXNzZXMgdGhlIGZyYW1lIGFuZCBDaGVja3Mga25vd24gb3IgdW5rbm93blxuQVdTIGFuZCBEYXRhYmFzZSAtLT4-IExvZyBFbnRyeSA6IEtub3duICBcbkFXUyBhbmQgRGF0YWJhc2UgLS1YIFNlY3VyaXR5IDogVW5rbm93biBOb3RpZmllZFxuU2VjdXJpdHkgLT4-IFBlcnNvbiA6IFNlY3VyaXR5IENoZWNrXG5QZXJzb24gLS0-PiBBV1MgYW5kIERhdGFiYXNlOiBDaGVja2luZyBjbGVhciBhbmQgUmVnaXN0cmF0aW9uIGRvbmVcbkFXUyBhbmQgRGF0YWJhc2UgLS0-PiBMb2cgRW50cnkgOiBFbnRyeSBvZiB0aGUgbmV3IHJlZ2lzdHJhdGlvblxuTG9nIEVudHJ5IC0-PiBTZWN1cml0eSA6IE5ldyBFbnRyeSBOb3RpZmllZFxuIiwibWVybWFpZCI6eyJ0aGVtZSI6ImRlZmF1bHQifX0)](https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG5QZXJzb24gLT4-IENhbWVyYTogRmV0Y2hlcyBldmVyeSBmcmFtZVxuQ2FtZXJhIC0tPiBBV1MgYW5kIERhdGFiYXNlOiBQcm9jZXNzZXMgdGhlIGZyYW1lIGFuZCBDaGVja3Mga25vd24gb3IgdW5rbm93blxuQVdTIGFuZCBEYXRhYmFzZSAtLT4-IExvZyBFbnRyeSA6IEtub3duICBcbkFXUyBhbmQgRGF0YWJhc2UgLS1YIFNlY3VyaXR5IDogVW5rbm93biBOb3RpZmllZFxuU2VjdXJpdHkgLT4-IFBlcnNvbiA6IFNlY3VyaXR5IENoZWNrXG5QZXJzb24gLS0-PiBBV1MgYW5kIERhdGFiYXNlOiBDaGVja2luZyBjbGVhciBhbmQgUmVnaXN0cmF0aW9uIGRvbmVcbkFXUyBhbmQgRGF0YWJhc2UgLS0-PiBMb2cgRW50cnkgOiBFbnRyeSBvZiB0aGUgbmV3IHJlZ2lzdHJhdGlvblxuTG9nIEVudHJ5IC0-PiBTZWN1cml0eSA6IE5ldyBFbnRyeSBOb3RpZmllZFxuIiwibWVybWFpZCI6eyJ0aGVtZSI6ImRlZmF1bHQifX0)
+
+### EXIT
+[![](https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG5QZXJzb24gLT4-IENhbWVyYTogRmV0Y2hlcyBldmVyeSBmcmFtZVxuQ2FtZXJhIC0tPiBBV1MgYW5kIERhdGFiYXNlOiBQcm9jZXNzZXMgdGhlIGZyYW1lIGFuZCBDaGVja3Mgd2hvIHRoZSBwZXJzb24gaXNcbkFXUyBhbmQgRGF0YWJhc2UgLS0-PiBMb2cgRW50cnkgOiBFeGl0IHRpbWUgb2YgdGhlIHBlcnNvbiB1cGRhdGVkIGF1dG9tYXRpY2FsbHkgICIsIm1lcm1haWQiOnsidGhlbWUiOiJkZWZhdWx0In19)](https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG5QZXJzb24gLT4-IENhbWVyYTogRmV0Y2hlcyBldmVyeSBmcmFtZVxuQ2FtZXJhIC0tPiBBV1MgYW5kIERhdGFiYXNlOiBQcm9jZXNzZXMgdGhlIGZyYW1lIGFuZCBDaGVja3Mgd2hvIHRoZSBwZXJzb24gaXNcbkFXUyBhbmQgRGF0YWJhc2UgLS0-PiBMb2cgRW50cnkgOiBFeGl0IHRpbWUgb2YgdGhlIHBlcnNvbiB1cGRhdGVkIGF1dG9tYXRpY2FsbHkgICIsIm1lcm1haWQiOnsidGhlbWUiOiJkZWZhdWx0In19) 
+
+
+## Solution Flowchart
+
+### ENTRY
+
+[![](https://mermaid.ink/img/eyJjb2RlIjoiZ3JhcGggTFJcbkFbUGVyc29uXSAtLUNhbWVyYSBjYXB0dXJlcyBhbmQgZGV0ZWN0cyAtLT4gQigoS25vd24pKVxuQS0tQ2FtZXJhIGNhcHR1cmVzIGFuZCBkZXRlY3RzIC0tPiBDKChVbmtub3duKSlcbkIgLS0-IEQoQWxsb3dzIEVudHJ5KVxuRCAtLT4gR3tEYXRhYmFzZSBMb2cgRW50cnl9XG5DIC0tPiBFKFNlY3VyaXR5IENoZWNrKVxuRSAtLUFsbG93cy0tPiBGKChSZWdpc3RlcikpXG5GIC0tPiBLKChBbGxvd3MgRW50cnkpKVxuSyAtLT4gSHtEYXRhYmFzZSBMb2cgRW50cnl9XG5HIC0tPiBJW1NlY3VyaXR5IE9mZmljZV1cbkggLS0-IEkiLCJtZXJtYWlkIjp7InRoZW1lIjoiZGVmYXVsdCJ9fQ)](https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoiZ3JhcGggTFJcbkFbUGVyc29uXSAtLUNhbWVyYSBjYXB0dXJlcyBhbmQgZGV0ZWN0cyAtLT4gQigoS25vd24pKVxuQS0tQ2FtZXJhIGNhcHR1cmVzIGFuZCBkZXRlY3RzIC0tPiBDKChVbmtub3duKSlcbkIgLS0-IEQoQWxsb3dzIEVudHJ5KVxuRCAtLT4gR3tEYXRhYmFzZSBMb2cgRW50cnl9XG5DIC0tPiBFKFNlY3VyaXR5IENoZWNrKVxuRSAtLUFsbG93cy0tPiBGKChSZWdpc3RlcikpXG5GIC0tPiBLKChBbGxvd3MgRW50cnkpKVxuSyAtLT4gSHtEYXRhYmFzZSBMb2cgRW50cnl9XG5HIC0tPiBJW1NlY3VyaXR5IE9mZmljZV1cbkggLS0-IEkiLCJtZXJtYWlkIjp7InRoZW1lIjoiZGVmYXVsdCJ9fQ)
+
+### EXIT
+
+[![](https://mermaid.ink/img/eyJjb2RlIjoiZ3JhcGggTFJcbkFbUGVyc29uXSAtLUNhbWVyYSBjYXB0dXJlcyBhbmQgZGV0ZWN0cyAtLT4gQigoS25vd24pKVxuQiAtLT4gQ1tVcGRhdGVzIHRoZSBMb2cgd2l0aCB0aGUgZGV0ZWN0ZWQgcGVyc29uJ3MgZXhpdCB0aW1lXSAgIiwibWVybWFpZCI6eyJ0aGVtZSI6ImRlZmF1bHQifX0)](https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoiZ3JhcGggTFJcbkFbUGVyc29uXSAtLUNhbWVyYSBjYXB0dXJlcyBhbmQgZGV0ZWN0cyAtLT4gQigoS25vd24pKVxuQiAtLT4gQ1tVcGRhdGVzIHRoZSBMb2cgd2l0aCB0aGUgZGV0ZWN0ZWQgcGVyc29uJ3MgZXhpdCB0aW1lXSAgIiwibWVybWFpZCI6eyJ0aGVtZSI6ImRlZmF1bHQifX0)
+
+
+## Use Cases 
+
+[![](https://mermaid.ink/img/eyJjb2RlIjoiZ3JhcGggVEJcbkEoSFVNQU4gQVVUSEVOVElDQVRJT04gU1lTVEVNKSAtLT4gQigoU0NIT09MUykpXG5BLS0-QygoQ09MTEVHRVMvIFVOSVZFUlNJVElFUykpXG5BLS0-RCgoT0ZGSUNFUykpXG5BLS0-RSgoUkVTSURFTlRJQUwgQ09NUExFWEVTKSlcbkEtLT5GKChCSUcgTUFOU0lPTlMpKSIsIm1lcm1haWQiOnsidGhlbWUiOiJkZWZhdWx0In19)](https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoiZ3JhcGggVEJcbkEoSFVNQU4gQVVUSEVOVElDQVRJT04gU1lTVEVNKSAtLT4gQigoU0NIT09MUykpXG5BLS0-QygoQ09MTEVHRVMvIFVOSVZFUlNJVElFUykpXG5BLS0-RCgoT0ZGSUNFUykpXG5BLS0-RSgoUkVTSURFTlRJQUwgQ09NUExFWEVTKSlcbkEtLT5GKChCSUcgTUFOU0lPTlMpKSIsIm1lcm1haWQiOnsidGhlbWUiOiJkZWZhdWx0In19)

server.py → face_recognition_server/server.py

@@ -24,7 +24,7 @@ def upload_data():
         if file.filename == '' and name == '':
             return redirect(request.url)
 
-        if file and allowed_file(file.filename) and name:
+        if file and name:
             # The image file seems valid! Detect faces and return the result.
             return add_new_face(file,name)
 
@@ -74,7 +74,7 @@ def detect_faces_in_image(file_stream):
 
     for (face_encoding,face_location) in zip(unknown_face_encodings,unknown_face_locations):
         # See if the face is a match for the known face(s)
-        matches = face_recognition.compare_faces(known_encodings, face_encoding)
+        matches = face_recognition.compare_faces(known_encodings, face_encoding,tolerance=0.5)
         index = -1
         name = ""
         found = False

gates_server/entry_server.py

@@ -0,0 +1,32 @@
+import requests
+import cv2
+import time
+import json
+
+entry = True
+gate_no = 1
+
+video_capture = cv2.VideoCapture(0)
+
+process_frame = True
+
+while True:
+    if process_frame:
+        ret, frame = video_capture.read()
+        cv2.imwrite("entry_frame.jpg",frame)
+        entry_res = requests.post("http://192.168.43.100:5001/process",files={"file":open('entry_frame.jpg','rb')})
+        entry_res_json = json.loads(entry_res.text)
+        try:
+            if entry_res_json[0]:
+                for data in entry_res_json:
+                    if data["face_matched"] and data["index"]:
+                        r = requests.get(f"http://192.168.43.100:5010/check-entry/{data['index']}/{entry}/{gate_no}")
+                        if r.text == "True":
+                            print(f"Welcome {data['name']}")
+                        else:
+                            print(f"Welcome Back {data['name']}")
+                    elif data["face_matched"] == False:
+                        r = requests.post(f"http://192.168.43.100:5010/new-user/{gate_no}",files={"file":open('entry_frame.jpg','rb')}) 
+                        print("Unknown")
+        except:
+            continue

gates_server/exit_server.py

@@ -0,0 +1,34 @@
+from flask import Flask, render_template, Response
+import requests
+import cv2
+import time
+import json
+
+entry = False
+gate_no = 1
+
+video_capture = cv2.VideoCapture(0)
+
+known_captured_index = []
+
+process_frame = True
+
+while True:
+    if process_frame:
+        ret, frame = video_capture.read()
+        cv2.imwrite("exit_frame.jpg",frame)
+        exit_res = requests.post("http://192.168.43.100:5001/process",files={"file":open('exit_frame.jpg','rb')})
+        exit_res_json = json.loads(exit_res.text)
+        try:
+            if exit_res_json[0]:
+                for data in exit_res_json:
+                    if data["face_matched"] and data["index"]:
+                        r = requests.get(f"http://192.168.43.100:5010/check-exit/{data['index']}/{entry}/{gate_no}")
+                        if r.text == "True":
+                            print(f"Bye {data['name']}")
+                        else:
+                            print(f"Bye Again {data['name']}")
+                    # elif data["face_matched"] == False:
+                        # print('Unknown')
+        except:
+            continue

security_server/database.py

@@ -0,0 +1,68 @@
+from pymongo import MongoClient
+import datetime
+
+db = None
+log = None
+
+client = MongoClient('mongodb://192.168.43.100:27017')
+
+db = client["face_recognition"]
+log = db["log"]
+user = db["user"]
+unknown = db["unknown"]
+
+print("Database Started")
+
+#Log
+def insert_log(data,action,gate):
+    temp = {
+        "registered": True,
+        "index": data,
+        "time": datetime.datetime.now(),
+        "entry": action,
+        "gate": gate,
+        "done": False
+    }
+    x = log.insert_one(temp)
+    print("[DATABASE] Person Entry")
+
+def exit_log(data,action,gate):
+    temp = {
+        "registered": True,
+        "index": data,
+        "time": datetime.datetime.now(),
+        "entry": action,
+        "gate": gate,
+        "done": True
+    }
+    x = log.insert_one(temp)
+    print("[DATABASE] Person exit")
+
+def update_log(index,name):
+    query = {"name":name,"index":index}
+    update = { "$set": {"done": True}}
+    log.update_one(query,update)
+
+def check_log(index,action,done):
+    query = {"index":index, "entry": action,"done":done}
+    data = log.find(query)
+    print("[DATABASE] Person Check")
+    if data[0]:
+        print("Inside")
+        return True
+    else:
+        print("Outside")
+        return False
+
+#user
+def add_unknown_user(data):
+    unknown.insert_one(data)
+    print("[DATABASE] Unknown User created")
+
+def delete_unknown_user(data):
+    unknown.delete_one(data)
+    print("[DATABASE] Unknown user deleted")
+
+def add_known_user(data):
+    user.insert_one(data)
+    print("[DATABASE] New user created")

security_server/security_server.py

@@ -0,0 +1,92 @@
+from flask import Flask, jsonify, request, redirect, url_for, render_template
+import requests
+import cv2
+import time
+import json
+import database as db
+import random
+app = Flask(__name__)
+
+entered_indexes = []
+unknown_detected = []
+
+@app.route('/index')
+def index():
+    return render_template('index.html', entered=len(entered_indexes), unknown_detected=len(unknown_detected))
+
+@app.route('/entry/<index>/<entry>/<gate>',methods=['GET'])
+def new_entry(index,entry,gate):
+    entered_indexes.append(index)
+    db.insert_log(index,entry,gate)
+    return index
+
+@app.route('/exit/<index>/<entry>/<gate>')
+def exit_entry(index,entry,gate):
+    entered_indexes.remove(index)
+    db.exit_log(index,entry,gate)
+    return index
+
+@app.route('/check-entry/<index>/<entry>/<gate>')
+def check_entry(index,entry,gate):
+    if index not in entered_indexes:
+        i = new_entry(index,entry,gate)
+        return "True"
+    else:
+        return "False"
+
+@app.route('/check-exit/<index>/<entry>/<gate>')
+def check_exit(index,entry,gate):
+    if index in entered_indexes:
+        i = exit_entry(index,entry,gate)
+        return "True"
+    else:
+        return "False"
+
+@app.route('/new-user/<gate>',methods=['POST'])
+def add_user(gate):
+    if request.method == 'POST':
+        if 'file' not in request.files:
+            return redirect(request.url)
+
+        file = request.files['file']
+        if file.filename == '':
+            return redirect(request.url)
+
+        if file and gate not in unknown_detected:
+            # The image file seems valid! Detect faces and return the result.
+            print(f"Unknown at gate {gate}")
+            unknown_detected.append(gate)
+            file.save(f"Unknown-{gate}.jpg")
+            data = {
+                "gate": gate,
+                "image": f"Unknown-{gate}.jpg"
+            }
+            db.add_unknown_user(data)
+            return "Unknown User"
+        else:
+            return "Wait for registration"
+
+def delete_unknown(gate):
+    unknown_detected.remove(gate)
+    data = {
+        "gate": gate,
+        "image": f"Unknown-{gate}.jpg"
+    }
+    db.delete_unknown_user(data)
+
+@app.route('/add-new-user',methods=['POST'])
+def add_new_user():
+    name = request.form.get('name')
+    gate = request.form.get('gate')
+    res = requests.post("http://192.168.43.100:5001/add",files={"file":open(f"Unknown-{gate}.jpg",'rb')},data={"name":name})
+    res_json = json.loads(res.text)
+    # user = {
+    #     "name": name,
+    #     "index": res_json['id']
+    # }
+    # db.add_known_user(user)
+    delete_unknown(gate)
+    return res.text
+
+if __name__ == '__main__':
+    app.run(host='0.0.0.0', debug=True,port=5010)

security_server/templates/index.html

@@ -0,0 +1,43 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <meta http-equiv="X-UA-Compatible" content="ie=edge">
+    <link rel="stylesheet" href="https://bootswatch.com/4/cyborg/bootstrap.css">
+    <title>Security</title>
+</head>
+<body>
+
+    <div class="container">
+        <div class="mx-auto text-center">
+            <h2 class="text-secondary">Live Status</h2>
+            <div class="row justify-content-center">
+                <div class="col-md-4">
+                    <div class="card text-white bg-success mb-3" style="max-width: 20rem;">
+                    <div class="card-header">No of guests visitors</div>
+                    <div class="card-body">
+                        <h4 class="card-title">{{entered}}</h4>
+                    </div>
+                    </div>
+                </div>
+                <div class="col-md-4">
+                    <div class="card text-white bg-warning mb-3" style="max-width: 20rem;">
+                    <div class="card-header">No of total visitors</div>
+                    <div class="card-body">
+                        <h4 class="card-title">{{unknown_detected}}</h4>
+                    </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+        <div>
+
+        </div>
+    </div>
+
+    <script src="https://code.jquery.com/jquery-3.4.1.slim.min.js" integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous"></script>
+    <script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js" integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous"></script>
+    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js" integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous"></script>
+</body>
+</html>