An AR+ML prototype that enables reducing parts from your environment and pasting them in a image enhancing application.
Even even though most effective Photoshop is being dealt with currently, it’ll additionally handle assorted outputs in the long run.
Demo & extra infos: Thread
Modules
This prototype runs as 3 self sustaining modules:
-
The cell app
- Take a look at out the /app folder for instructions on easy methods to deploy the app to your cell.
-
The local server
- The interface between the cell app and Photoshop.
- It finds the location pointed on display cloak by the digicam the usage of screenpoint
- Take a look at out the /server folder for instructions on configuring the local server
-
The object detection / background elimination service
- For now, the salience detection and background elimination are delegated to an exterior service
- It’d be loads extra efficient to make use of one thing love DeepLap straight throughout the cell app. Nonetheless that hasn’t been applied on this repo yet.
Utilization
1 – Configure Photoshop
- Trot to “Preferences > Poke-ins”, enable “Some distance away Connection” and build a friendly password that you just are going to be wanting later.
2 – Setup the outdoor salience object detection service
-
As mentioned above, for the time being, you might want to deploy the
BASNet model (Qin & al, CVPR 2019) as an exterior HTTP service the usage of this BASNet-HTTP wrapper (requires a CUDA GPU) -
Potentialities are you’ll presumably well need the deployed service URL to configure the local server
3 – Configure and escape the local server
- Apply the instructions in /server to setup & escape the local server.
4 – Configure and escape the cell app
- Apply the instructions in /app to setup & deploy the cell app.
Thanks and Acknowledgements
- BASNet code for ‘BASNet: Boundary-Mindful Salient Object Detection code‘, Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan and Martin Jagersand
- RunwayML for the Photoshop paste code
