An AR+ML prototype that allows cutting elements from your surroundings and pasting them in an image editing software. Although only Photoshop is being handled currently, it may handle different...

An AR+ML prototype that enables reducing parts from your environment and pasting them in a image enhancing application.

Even even though most effective Photoshop is being dealt with currently, it’ll additionally handle assorted outputs in the long run.

Demo & extra infos: Thread

⚠️ That is a research prototype and never a consumer / photoshop client application.


This prototype runs as 3 self sustaining modules:

  • The cell app

    • Take a look at out the /app folder for instructions on easy methods to deploy the app to your cell.
  • The local server

    • The interface between the cell app and Photoshop.
    • It finds the location pointed on display cloak by the digicam the usage of screenpoint
    • Take a look at out the /server folder for instructions on configuring the local server
  • The object detection / background elimination service

    • For now, the salience detection and background elimination are delegated to an exterior service
    • It’d be loads extra efficient to make use of one thing love DeepLap straight throughout the cell app. Nonetheless that hasn’t been applied on this repo yet.


1 – Configure Photoshop

  • Trot to “Preferences > Poke-ins”, enable “Some distance away Connection” and build a friendly password that you just are going to be wanting later.

2 – Setup the outdoor salience object detection service

  • As mentioned above, for the time being, you might want to deploy the
    BASNet model (Qin & al, CVPR 2019) as an exterior HTTP service the usage of this BASNet-HTTP wrapper (requires a CUDA GPU)

  • Potentialities are you’ll presumably well need the deployed service URL to configure the local server

3 – Configure and escape the local server

  • Apply the instructions in /server to setup & escape the local server.

4 – Configure and escape the cell app

  • Apply the instructions in /app to setup & deploy the cell app.

Thanks and Acknowledgements

Read Extra

Internet of Things
No Comment

Leave a Reply