Re: About SQL Backend Project (GSoC)

[Martin Husemann <martin%duskware.de@localhost>]

On Tue, Mar 13, 2018 at 11:54:33AM +0000, Nizar Benshaqi wrote:
> Hello,
> 
> My name is Nizar Benshaqi , 19 Years old from Morocco,
> 
> I'm really interested in a NetBSD GSoC project: Creating an SQL backend and
> statistics/query page for ATF test results (
> https://wiki.netbsd.org/projects/project/atf-sql-backend/)

Hello Nizar!

> I am an experienced web developer , with knowledge in PostgreSQL , I can
> build such project using either PHP , Node.JS or Python ( Depending on your
> requirements) for the backend.

PostgreSQL and Python sound good.

> As for the tool to read the ATF xml data and insert it into the database, I
> can build it either using Python , or remotely using PHP (Depending on the
> file size).

Preferably Python again here...

> If you can provide me with ATF xml data I will provide you with more
> in-depth details on how to implement and build these tools.

A good start is probably to look here:

 - the list of all currently running tests:

	http://releng.netbsd.org/test-results.html

If you look at the upper (babylo5) tests, you can see what we get currently,
e.g. for i386 this month:

	http://releng.netbsd.org/b5reports/i386/commits-2018.03.html#end

If you scroll a bit up, you'll note that at 2018.03.13.02.24.26 a lot of
ipsec tests stopped failing. This is exactly the kind of information we
would want to search within the database.

If you look at individual test lines, you can see the html output,
like:

	http://releng.netbsd.org/b5reports/i386/2018/2018.03.13.02.24.26/test.html

or you can download the raw or xml data:

	http://releng.netbsd.org/b5reports/i386/2018/2018.03.13.02.24.26/test.tps
	http://releng.netbsd.org/b5reports/i386/2018/2018.03.13.02.24.26/test.xml

Your tool should be able to process one of those formats, so I guess
xml will be the simple way.


Assume that the test data will come from different enviroments, so we will
need a bit of meta data, but that is trivial (and mostly static per test
environment, with rare changes every year or so, e.g. when the qemu version
gets updated).


Martin