Azure Custom Vision tanpa Azure, atau "di mana topeng mereka?" Bagaimana kita mengenali topeng di wajah (dan bagian tubuh lainnya)

Di antara sekumpulan contoh untuk Azure di GitHub , satu hal yang sangat menarik ditemukan: pengenalan gambar di Raspberry Pi, offline. Penulis mengusulkan untuk menyiapkan model pembelajaran mesin di salah satu layanan cloud Azure, kemudian mentransfernya ke komputer yang sebagian besar tidak memiliki koneksi Internet, setelah itu pengenalan pola akan berfungsi secara offline. Pengembang telah menyiapkan proyek untuk dua platform: ARM32 (Raspberry Pi sendiri) dan AMD64 (tetapi tanpa dukungan webcam).

Setelah hype virus corona, kami memutuskan untuk mengadaptasi contoh untuk mengenali apakah seseorang memakai topeng atau tidak. Segera menjadi jelas bahwa langkah-langkah untuk menyesuaikan contoh tersebut tidak sepenuhnya jelas. Dalam materi ini, kami akan mempertimbangkan secara berurutan bagaimana contoh ini, pertama, dapat dimodifikasi agar berfungsi dengan model pembelajaran mesin kami, dan kedua, dapat ditransfer ke platform AMD64. Mari kita asumsikan bahwa komputer kita, di mana model dijalankan, mengontrol pintu putar: jika seseorang datang memakai topeng, pintu putar terbuka, jika tidak, tetap tertutup.

Ini rumit...

Hampir setiap penyedia cloud memiliki seperangkat alat pembelajaran mesin, dan hambatan untuk masuk seringkali rendah.

Azure Computer Vision, , : API, . Azure Custom Vision, , . , . - , , .

, . , "" , :

• , - , ;
• ;
• .

, :

• ;
• , . , "" 100 ?

?

, " ", " ". — . , " ". , Azure Custom Vision, , , .

: Azure IoT Edge . , :

• ;
• — ;
• , .. , ;

Azure IoT Edge

IoT Edge . , IoT Edge — Linux, "" , , , Docker- . IoT Edge Azure IoT Hub. IoT Edge , , Azure, IoT Edge , .

"" IoT Edge Linux, 2019 IoT Edge Windows 10 Enterprise LTSC — .

IoT Edge , "" Azure:

• Azure Functions ( )
• Azure Stream Analytics ( )
• Azure Machine Learning ( "")
• Azure Custom Vision ( )
• Azure SQL Server ( "")

Custom Vision. "" .

IoT Edge . , (), Microsoft HTTP, .. web-, .

Microsoft

. , , .

-, USB , IoT Edge.

IoT Edge? , , IoT Hub, ( ) IoT Hub. "" .

IoT Hub. IoT Edge "" , . , Container Registry ( ).

, , :

• Camera: . "" ( ! ) HTTP ;
• AI: , . , , "" . Custom Vision;
• Display: , . , , , ( ).

, IoT Hub.

- , . , - Raspberry Pi , .

, . AMD64 , - - , , . , "" -, ARM32.

. AMD64 "" SenseHat, Raspberry Pi, , SenseHat, , - .

(Custom Vision)

Custom Vision ( Azure) New Project.

• Name — ;
• Description — ;
• Resource — Create new Cognitive Services;
• Classification Types — Multilabel, / , — ( " ")
• Domains — General (compact).

:

• , MaskOnFace;
• (" "), MaskNotOnFace — - , (, -, );
• , , (Negative).

.

Train, Advanced training , . Performace .

, , .

Probability Threshold — , . Probability Threshold = 90% , , 90%. , . , (/), "" " " "" "". , - " 89%". , "" "". Probability Threshold 90%, " 89%" "", " 91%" "".

, , "" "". , , . , .

• Recall () ;
• Precision () — .

— .

, .

(Publish), (Export — Dockerfile — Linux). IoT Edge.

- ( — ) AVX (, AVX , , grep avx /proc/cpuinfo) — tensorflow. AVX, , . . Intel NUC Core i5.

Ubuntu 20.04 LTS. , IoT Edge ( Microsoft?), .

, Edge. (~).

:

sudo apt-get update
sudo apt-get install wget nano
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

:

wget https://packages.microsoft.com/config/ubuntu/20.04/prod.list
mv prod.list microsoft-prod.list
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

sources.list.d, "" Microsoft:

sudo cp ./microsoft-prod.list /etc/apt/sources.list.d/
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Microsoft GPG:

wget https://packages.microsoft.com/keys/microsoft.asc
cat microsoft.asc | gpg --dearmor > microsoft.gpg
sudo cp ./microsoft.gpg /etc/apt/trusted.gpg.d/
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

( — , apt " "):

sudo apt-get update
sudo apt-get install moby-engine
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

IoT Edge:

sudo apt-get install iotedge
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, ( Ubuntu 20.04 — ), . Ubuntu (18.04 — , , ) (, , ). GitHub. Latest release dpkg. libssl :

wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl1.0/libssl1.0.0_1.0.2n-1ubuntu5_amd64.deb
sudo dpkg -i libssl1.0.0_1.0.2n-1ubuntu5_amd64.deb
wget https://github.com/Azure/azure-iotedge/releases/download/1.0.10.4/libiothsm-std_1.0.10.4-1_ubuntu16.04_amd64.deb
sudo dpkg -i libiothsm-std_1.0.10.4-1_ubuntu16.04_amd64.deb
wget https://github.com/Azure/azure-iotedge/releases/download/1.0.10.4/iotedge_1.0.10.4-1_ubuntu16.04_amd64.deb
sudo dpkg -i iotedge_1.0.10.4-1_ubuntu16.04_amd64.deb
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

IoT Edge , Azure. , :

sudo nano /etc/iotedge/config.yaml
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, Azure device_connection_string provisioning. , , IoT Edge Azure.

Azure

IoT Edge IoT Hub, Azure. — .

IoT Hub IoT Edge . , , :

• IoT Hub;
• IoT Edge;
• ;

IoT Edge IoT Hub, .

IoT Edge :

• . -, USB. HTTP;
• , ( / / ). HTTP, ;
• (-). -, .

:

, , .

! Windows 10, IoT Edge.

• Visual Studio Code. Visual Studio;
• Azure IoT Edge Extension. Select IoT Hub, IoT Hub;
• Azure IoT Tools;
• Python 3.8.3. Add Python to PATH — Task 4;
• , , , Hyper-V . — , OK ;
• Docker for Windows — Task 5;
• Git Windows .

! Docker : bcdedit /set {current} hypervisorlaunchtype Auto .



! Hyper-V VirtualBox. , VirtualBox, : bcdedit /set {current} hypervisorlaunchtype off, "" Hyper-V Docker.

, :

git clone https://github.com/Azure-Samples/Custom-vision-service-iot-edge-raspberry-pi.git
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, :

git checkout 6b3540f9b31121321f9e75d8df0ed86397c9324f
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Visual Studio Code (File — Open Folder) . , . , .

, , . , , , . , , .

. — . — , , .

, .

.env

() , Azure:

• CONTAINER_REGISTRY_ADDRESS="__.azurecr.io"
• CONTAINER_REGISTRY_USERNAME="_"
• CONTAINER_REGISTRY_PASSWORD=""

deployment.template.json

• modules — camera-capture — env — RESIZE_WIDTH — 640, RESIZE_HEIGHT — 480
• modules — camera-capture — settings — image — ${MODULES.CameraCapture.amd64}
• modules — sensehat-display — env — THRESHOLD — value — 0.9
• modules — sensehat-display — settings — image — ${MODULES.SenseHatDisplay.amd64}, createOptions — HostConfig "PortBindings": { "8000/tcp": [ { "HostPort": "8000" } ] }
• modules — image — settings — image — ${MODULES.ImageClassifierService.amd64}

deployment.template_AMD64

.

ameraCapture\amd64.Dockerfile

AMD64 -, , , . .

CameraCapture.py

( Raspberry Pi), , . time.sleep(1.0) 1.0 0.1.

ImageClassifierService\amd64.Dockerfile

AMD64. .

labels.txt

, . Custom Vision, :

MaskNotOnFace
MaskOnFace
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

model.pb

Custom Vision — .

SenseHatDisplay\amd64.Dockerfile

, .

SenseHatDisplay — . AMD64 SenseHat , , , , SenseHat, , - Python, .

.

SenseHatDisplay\module.json

platform , amd64:

"amd64": "./amd64.Dockerfile",
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

SenseHatDisplay\app ()

, . -, .

• custom.js
• index.htm
• jquery.js
• style.css
• DisplayManager.py
• MessageParser.py

- "". , :

• : IoT Edge, http://127.0.0.1:5012;
• — http://127.0.0.1:8000/status.

, - — , . , , . http://127.0.0.1:8000.

.

• VS Code View — Command Pallette — Azure IoT Edge: Set Default Target Platform for IoT Edge Solution amd64;
• Azure IoT Edge: Build and Push IoT Edge Solution. , IoT Edge;
• IoT Edge, config/deployment.json Create Deployment for Single device, , ;
• ( -!). IoT Edge Extension Start Monitoring D2C Message, .

IoT Edge UI - http://localhost:8000, -, . , , .

— , .

, , MaskNotOnFace, , .

- ...

, , IoT Edge . ( , , , AVX / tensorflow ). IoT Edge . . , :

• : iotedge list. ( );
  
  
• : iotedge logs _. , .

, IoT Edge, , ( Azure), IoT Edge .

?

IoT Edge , .

, , IoT Edge, , , , , " " - SQL Edge, .

IoT Edge Azure, .

— , , — .

, Azure , , , .

Microsoft — .

.

— , . sergant (at) quarta.ru.