Community

Use Travis to extract testing APKs for PSLab

Post author:CloudyPadmal
Post published:August 21, 2018
Post category:Community Pocket Science Lab Tutorial

Travis CI is a popular continuous integration tool built to test software and deployments done at GitHub repositories. They provide free plans to open source projects. PSLab Android is using Travis to ensure that all the pull requests made and the merges are build-bug frees. Travis can do this pretty well, but that is not all it can do. It’s a powerful tool and we can think of it as a virtual machine (with high specs) installed in a server for us to utilize efficiently.

There was a need in PSLab development that, if we want to test the functionalities of code at a branch. To fulfil this need, one has to download the branch to his local machine and use gradle to build a new apk. This is a tedious task, not to mention reviewers, even developers wouldn’t like to do this.

If the apk files were generated and in a branch, we can simply download them using a console command.

$ wget https://raw.github.com/fossasia/<repository>/<branch>/<file with extension>

With the help of Travis integration, we can create a simple script to generate these apks for us. This is done as follows;

Once the Travis build is complete for a triggering event such as a pull request, it will run it’s “after_success” block. We are going to execute a script at this point. Simply add the following code snippet.

after_success:
  - 'if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then bash <script-name>.sh; fi'

This will run the script you have mentioned using bash. Here we will have the following code snippets in the specified bash script.

First of all we have to define the branch we want to build. This can be done using a variable assignment.

export DEVELOPMENT_BRANCH=${DEVELOPMENT_BRANCH:-development}

Once the build is complete, there will be new folders in the virtual machine. One of them is the app folder. Inside this folder contains the build folder where all the apk files are generated. So the next step is to copy these apk files to a place of our preference. I am using a folder named apk to make it much sense.

cd apk
\cp -r ../app/build/outputs/apk/*/**.apk .
\cp -r ../app/build/outputs/apk/debug/output.json debug-output.json
\cp -r ../app/build/outputs/apk/release/output.json release-output.json
\cp -r ../README.md .

Usually, the build folder has following apk files

app-debug.apk
app-release-unsigned.apk
app-release.apk

Release apks are usually signed with a key and it would cause issues while installation. So we have to filter out the debug apk that we usually use for debugging and get it as the output apk. This involves simple file handling operations in Linux and a bit of git.

First of all, rename the apk file so that it will be different from other files.

# Rename apks with dev prefixes
mv app-debug.apk app-dev-debug.apk

Then add and commit them to a specific branch where we want the output from.

git add -A
git commit -am "Travis build pushed [development]"
git push origin apk --force --quiet> /dev/null

Once it is all done, you will have a branch created and updated with the apk files you have defined.

Figure 1: UI of pslab-android apk branch

Reference:

https://travis-ci.org/

Adding Open Street Maps to PSLab Android

Post author:CloudyPadmal
Post published:August 21, 2018
Post category:Community Pocket Science Lab Tutorial

PSLab Android app is an open source app that uses fully open libraries and tools so that the community can use all it’s features without any compromises related to pricing or feature constraints. This will brings us to the topic how to implement a map feature in PSLab Android app without using proprietary tools and libraries. This is really important as now the app is available on Fdroid and they don’t allow apps to have proprietary tools in them if they are published there. In other words, it simply says we cannot use Google Maps APIs no matter how powerful they are in usage.

There is a workaround and that is using Open Street Maps (OSM). OSM is an open source project which is supported by a number of developers all around the globe to develop an open source alternative to Google Maps. It supports plenty of features we need in PSLab Android app as well. Starting from displaying a high resolution map along with caching the places user has viewed, we can add markers to show data points and locations in sensor data logging implementations.

All these features can be made available once we add the following dependencies in gradle build file. Make sure to use the latest version as there will be improvements and bug fixes in each newer version

implementation "org.osmdroid:osmdroid-android:$rootProject.osmVersion"
implementation "org.osmdroid:osmdroid-mapsforge:$rootProject.mapsforgeVersion"
implementation "org.osmdroid:osmdroid-geopackage:$rootProject.geoPackageVersion"

OSM will be functional only after the following permission states were granted.

<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"  />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

In a view xml file, add the layout org.osmdroid.views.MapView to initiate the map view. There is a known issue in OSM library. That is during the initiation, if the zoom factor is set to a small value, there will be multiple instances of maps as shown in Fig 1. The solution is to have a higher zoom value when the map is loaded.

Figure 1: Multiple map tiles in OSM

Once we initialize the map view inside an activity, a zoom level can be easily set using a map controller as follows;

map = findViewById(R.id.osmmap);
map.setTileSource(TileSourceFactory.MAPNIK);
map.setBuiltInZoomControls(true);
map.setMultiTouchControls(true);

IMapController mapController = map.getController();
mapController.setZoom((double) 9);
GeoPoint startPoint = new GeoPoint(0.00, 0.00);
mapController.setCenter(startPoint);

After successfully implementing the map view, we can develop the business logic to add markers and descriptions to improve the usability of OS Maps. They will be available in the upcoming blog posts.

Reference:

Setting up environment to build PSLab Android app using Fdroid Build

Post author:CloudyPadmal
Post published:August 21, 2018
Post category:Android Community FOSSASIA GSoC Pocket Science Lab Tutorial

Fdroid is a place for open source enthusiasts and developers to host their Free and Open Source Software (FOSS) for free and get more people onboard into their community. In order to host an app in their repository, one has to go through a several steps of builds and tests. This is to ensure that the software provided by them are as quality and safe as they can ever be. They are not allowing proprietary libraries or tools to integrate into any app or they will be published outside the Fdroid main repository (fdroid-data) so that the users will know what they are downloading.

In a normal Linux computer where we are developing Android apps and have setup Android Studio will not be able to run the build command using:

$ fdroid build -v -l org.fossasia.pslab

The reason behind this is that we have not installed gradle and build tools required by the “fdroid build” because they are not useful in our day today activities for standalone activities. First thing we need to do is, install gradle separately. This will include adding gradle to $PATH as well.

Download the latest gradle version zip file or the version your project is using with the following command. In PSLab Android app, we are using 4.5.1 version and the snippet below include that version.

$ wget https://services.gradle.org/distributions/gradle-4.5.1-bin.zip

Next step is to install this in a local folder. We can select any path we want, but /opt/ folder is generally used in such scenarios.

sudo mkdir /opt/gradle
sudo unzip -d /opt/gradle gradle-4.5.1-bin.zip

Then we can add gradle to our $PATH variable using the following command:

$ export PATH=$PATH:/opt/gradle/gradle-4.5.1/bin

Now we are all set with gradle settings. Next step is to verify that the fdroid server is properly configured and up to date. When you run the build command after setting up the gradle in PC, it will throw an error similar to “failed to find any output apks”. This causes if the installed fdroid server version is old.

Fdroid server is running on python 3 and it will require some additional libraries pre-installed to properly function.

$ sudo apt-get install vagrant virtualbox git python3-certifi python3-libvirt python3-requestbuilder python3-yaml python3-clint python3-vagrant python3-paramiko python3-pyasn1 python3-pyasn1-modules

Once these libraries are installed, remove the previous instance of fdroidserver by using the following command:

$ sudo apt-get remove fdroidserver

Then we can reinstall the latest version of fdroid server from git using the following command:

$ git clone https://gitlab.com/fdroid/fdroidserver.git
export PATH="$PATH:$PWD/fdroidserver"

Now we are all set to do a brand new lint build on our PC to make our app ready to be published in Fdroid repository!

Reference:

Install gradle : https://www.vultr.com/docs/how-to-install-gradle-on-ubuntu-16-10
Gradle versions : https://gradle.org/releases
Setting up Fdroid-server : https://f-droid.org/en/docs/Build_Server_Setup/

Installing fdroidserver : https://gitlab.com/fdroid/fdroiddata/blob/master/README.md#quickstart

Building PSLab Android app with Fdroid

Post author:CloudyPadmal
Post published:June 24, 2018
Post category:Android Community FOSSASIA GSoC Pocket Science Lab Tutorial

Fdroid is a place for open source enthusiasts and developers to host their Free and Open Source Software (FOSS) for free and get more people onboard into their community. Hosting an app in Fdroid is not a fairly easy process just like hosting one in Google Play. We need to perform a set of build checks prior to making a merge request (which is similar to pull request in GitHub) in the fdroid-data GitLab repository. PSLab Android app by FOSSASIA has undergone through all these checks and tests and now ready to be published.

Setting up the fdroid-server and fdroid-data repositories is one thing. Building our app using the tools provided by fdroid is another thing. It will involve quite a few steps to get started. Fdroid requires all the apps need to be built using:

$ fdroid build -v -l org.fossasia.pslab

This will output a set of logs which tell us what went wrong in the builds. The usual one in a first time app is obviously the build is not taking place at all. The reason is our metadata file needs to be changed to initiate a build.

Build:<versioncode>,<versionname>
    commit=<commit which has the build mentioned in versioncode>
    subdir=app
    gradle=yes

When a metadata file is initially created, this build is disabled by default and commit is set to “?”. We need to fill in those blanks. Once completed, it will look like the snippet above. There can be many blocks of “Build” can be added to the end of metadata file as we are advancing and upgrading through the app. As an example, the latest PSLab Android app has the following metadata “Build” block:

Build:1.1.5,7
    commit=0a50834ccf9264615d275a26feaf555db42eb4eb
    subdir=app
    gradle=yes

In case of an update, add another “Build” block and mention the version you want to appear on the Fdroid repository as follows:

Auto Update Mode:Version v%v
Update Check Mode:Tags
Current Version:1.1.5
Current Version Code:7

Once it is all filled, run the build command once again. If you have properly set the environment in your local PC, build will end successfully assuming there were no Java or any other language syntax errors.

It is worth to mention few other facts which are common to Android software projects. Usually the source code is packed in a folder named “app” inside the repository and this is the common scenario if Android Studio builds up the project from scratch. If this “app” folder is one level below the root, that is “android/app”, the build instructions shown above will throw an error as it cannot find the project files.

The reason behind this is we have mentioned “subdir=app” in the metadata file. Change this to “subdir=android/app” and run the build again. The idea is to direct the build to find where the project files are.

Apart from that, the commit can be represented by a tag instead of a long commit hash. As an example, if we had merge commits in PSLab labeled as “v.<versioncode>”, we can simply use “commit=v.1.1.5” instead of the hash code. It is just a matter of readability.

Happy Coding!

Reference:

Metadata : https://f-droid.org/docs/Build_Metadata_Reference/#Build
PSLab Android app Fdroid : https://gitlab.com/fdroid/fdroiddata/merge_requests/3271/diffs

Publish an Open Source app on Fdroid

Post author:CloudyPadmal
Post published:June 9, 2018
Post category:Android Community FOSSASIA GSoC Pocket Science Lab Tutorial

Fdroid is a famous software repository hosted with numerous free and open source Android apps. They have a main repository where they allow developers hosting free and ad free software after a thorough check up on the app. This blog will tell you how to get your project hosted in their repository using steps I followed to publish the PSLab Android app.

Before you get started, make sure you have the consent from your developer community to publish their app on Fdroid. Fdroid requires your app to use all kind of open resources to implement features. If there is any closed source libraries in your app and you still want to publish it on Fdroid, you may have to reimplement that feature by any other mean without using closed source resources. They will also not allow to have Google’s proprietary “play-services” in your app along with proprietary ad services. You can find the complete inclusion policy document from their official page.

When your app is fully ready, you can get started with the inclusion procedure. Unlike how we are publishing apps on Google Play, publishing an app on Fdroid is as simple as sending a pull request to their main repository. That’s exactly what we have to do. In simple terms all we have to do is:

Fork the Fdroid main data repository
Make changes to their files to include our app
Do a pull request

First of all you need a GitLab account as the Fdroid repository is hosted in GitLab. Once you are ready with a GitLab account, fork and clone the f-droid-data repository. The next step is to install the fdroid-server. This can be simply done using apt:

$ sudo apt install fdroidserver

Once that is done, go into the directory where you cloned the repository and run the following command to check if the initiation is complete.

$ fdroid init

Then run the following command to read current meta data where it saves all the information related to existing apps on Fdroid;

$ fdroid readmeta

This will list out various details about the current meta files. Next step is to add our app details into this meta file. This can be done easily using following command or you can manually create folders and files. But the following is safer;

$ fdroid import --url https://github.com/fossasia/pslab-android --subdir app

Replace the link to repository from the –url tag in the above command. For instance the following will be the link for fossasia-phimpme android;

$ fdroid import --url https://github.com/fossasia/phimpme-android --subdir app

This will create a file named as “org.fossasia.pslab” in the metadata directory. Open up this text file and we have to fill in our details.

Categories
License
Web Site
Summary
Description

Description needs to be terminated with a newline and a dot to avoid build failures.

Once the file is filled up, run the following command to make sure that the metadata file is complete.

$ fdroid readmeta

Then run the following command to clean up the file

$ fdroid rewritemeta org.fossasia.pslab

We can automatically add version details using the following command:

$ fdroid checkupdates org.fossasia.pslab

Now run the lint test to see if the app is building correctly.

$ fdroid lint org.fossasia.pslab

If there are any errors thrown, fix them to get to the next step where we actually build the app:

$ fdroid build -v -l org.fossasia.pslab

Now you are ready to make the pull request which will then get reviewed by developers in Fdroid community to get it merged into their main branch. Make a commit and then push to your fork. From there it is pretty straightforward to make a pull request to the main repository. Once that is done, they will test the app for any insecurities. If all of them are passed, the app will be available in Fdroid!

Reference:

Quick Start: https://gitlab.com/fdroid/fdroiddata/blob/master/README.md#quickstart
Making merge requests: https://gitlab.com/fdroid/fdroiddata/blob/master/CONTRIBUTING.md#merge-requests

Implementing Clickable Images

Post author:CloudyPadmal
Post published:June 8, 2018
Post category:Android Community FOSSASIA GSoC Pocket Science Lab Tutorial

PSLab Android application is a feature rich compact app to user interface the PSLab hardware device. Similarly the PSLab device itself is a compact device with a plenty of features to replace almost all the analytical instruments in a school science lab. When a first time user takes the device and connect it with the Android app, there are so many pins labeled with abbreviations. This creates lots of complications unless the user checks the pinout diagram separately.

As a workaround a UI is proposed to integrate a layout containing the PSLab PCB image where user can click on each pin to get a dialog box explaining him what that specific pin is and what it does. This implementation can be done it two ways;

Using an Image map
Using (x,y) coordinates

The first implementation is more practical and can be applied with any device with any dimension. The latter requires some transformation to capture the correct position when user has clicked on a pin. So the first method will be implemented.

The idea behind using an image map is to have two images with exact dimensions on top of each other. The topmost image will be the color map which we create ourselves using unique colors at unique heat points. This image will have the visibility setting invisible as the main idea is to let the user see a meaningful image and capture the positions using a secondary in the back end.

To make things much clear, let’s have a look at a color map image I am suggesting here for a general case.

If we overlap the color map with the PSLab layout, we will be able to detect where user has clicked using Android onTouchEvent.

@Override
public boolean onTouchEvent(MotionEvent ev) {
   final int action = ev.getAction();
   final int evX = (int) ev.getX();
   final int evY = (int) ev.getY();
   switch (action) {
       case MotionEvent.ACTION_UP :
         int touchColor = getHotspotColor (R.id.backgroundMap, evX, evY);
         /* Display the relevant pin description dialog box here */
         break;
   }
   return true;
}

Color of the clicked position can be captured using the following code;

public int getHotspotColor (int hotspotId, int x, int y) {
   ImageView img = (ImageView) findViewById (hotspotId);
   img.setDrawingCacheEnabled(true);
   Bitmap hotspots = Bitmap.createBitmap(img.getDrawingCache());
   img.setDrawingCacheEnabled(false);
   return hotspots.getPixel(x, y);
}

If we go into details, from the onTouchEvent we capture the (x,y) coordinates related to user click. Then this location is looked up for a unique color by creating a temporary bitmap and then getting the pixel value at the captured coordinate.

There is an error in this method as the height parameter always have an offset. This offset is introduced by the status bar and the action bar of the application. If we use this method directly, there will be an exception thrown out saying image height is less than the height defined by y.

Solving this issue involves calculating status bar and actionbar heights separately and then subtract them from the y coordinate.

Actionbar and status bar heights can be calculated as follows;

Rect rectangle = new Rect();
Window window = getWindow();
window.getDecorView().getWindowVisibleDisplayFrame(rectangle);
int statusBarHeight = rectangle.top;
int contentViewTop = window.findViewById(Window.ID_ANDROID_CONTENT).getTop();
int titleBarHeight= contentViewTop - statusBarHeight;

Using them, we can modify the captured coordinates as follows;

int touchColor = getHotspotColor (R.id.imageArea, evX, evY - statusBarHeight);

This way the exception is handled by adjusting the cursor position. Once this is done, it is all about displaying the correct pin description dialog box.

Reference:

Images with clickable areas: https://blahti.wordpress.com/2012/06/26/images-with-clickable-areas/

Calculate status bar height: https://stackoverflow.com/questions/3407256/height-of-status-bar-in-android