If it were me and I was intending to automate this I would probably do the following. Set up each test distro as a VirtualBox image and take a snapshot so I could easily roll back. Then I would write a script for each distro that downloaded the package, installed and launched the app. I would then probably query the window system to make sure the gui showed up, wait a period of time if I had to and take a screenshot.

This can probably all be done as a set of bash scripts.