Weekly Report #4: Shiny New Task

Weekly report for the fifth week of the Google Summer of Code

I have finished the basic implementation of the configure command, as detailed in my previous report, and I’ve started on adding static metadata support, i.e. moving arguments from setup.py into setup.cfg for easier parsing by external tools and eventual removal of setup.py scripts altogether.

To that end, I’ve read PEP 314 (Metadata 1.2) and PEP 390 (Static Metadata), copied docs from Python trunk and started editing them. My plan is to add a bit of doc, write tests that fail, add the code to make the test pass and the doc be true.

One part of PEP 390 bothers me: I’m supposed to add a distutils2.util.local_metadata function to read and interpret the new setup.cfg, but this was before the medatada module existed, so I think it’s better to put the new function in this module. A name starting with a verb is also a better idea for a function, e.g. distutils2.metadata.read_config. Since it always returns a DistributionMetadata instance, we could also just make it a classmethod (works on 2.4): DistributionMetadata.from_config. I’ll do that and move the code if told to :)

Posted in GSoC | Tagged , | Leave a comment

Weekly Report #3: Making Progress

Weekly report for the fourth week of the Google Summer of Code, sent to our mailing list last Thursday and copied here a bit late

A Milestone

Last week I have completed the configure command. It accepts the same options¹ as build and install and writes them to a file that is read back after configuration files and before command-line options.

(The –force option is renamed to –force-build and –force-install since it does not mean the same thing for both commands.)

You can read more info on my previous report, look at the code (or the tests). There is also user documentation.

Failing Tests Are Great

Writing tests was not as simple as expected, but still easy. Taking inspiration from existing tests, I tried using Distribution and configure objects directly, but couldn’t get them to work; I temporarily settled for running Python (sys.executable) in a subprocess. The second test I wrote helped me find a bug. Yay for tests!


I am one week late on my original schedule. The command is where I wanted it to be last Monday, yet it’s not entirely finished. It lacks tests, which I’ll add in the next weeks in parallel to other tasks, but there is another more important thing. As it is now, the configure command just records options without checking for conflicts (e.g. using both –prefix and –user for install). While not in the original description of the task, I think that it should be added. Prior art like 4Suite’s config command or a prototype for configure done by Tarek that I discovered only today support this idea. If you have an opinion on that, please express it in the comments on on Python #8254.

Even if I think the command still needs improvements and tests, its basic implementation is done, so I’m going to ask for review and pull. I’ll still work on it, but not exclusively. Now is the time for killing setup scripts!

Posted in GSoC | Tagged , | 2 Comments

Weekly Report #2: Short Week

Weekly report for the third week of the Google Summer of Code

We Have A configure Command…

I have been busier than expected this week-end and then tired, but I’ve finished the configure command on Thursday. A proof of concept (actually two successive different implementations) is available on one of my repos, but it needs further work.

I have three things to do in short delay about this command:

  1. explain its design and behavior in a blog post and ask for feedback;
  2. write documentation;
  3. write tests.

I’ll have to learn how to do tests that write, read and list files with distutils2.tests support code, or add some.

…But Don’t Use It Now

During the weekly meeting, Tarek helped me see clearly what needs to be done to make the command better. The code will get simpler, better and maybe smaller. I’ll write another entry about it.


If I’m done at the end of the week, or if I’m waiting for feedback, I’ll start on my next task: killing setup scripts.

I have some changesets unrelated to configure (various fixes) to be merged; I’ll ask via the relevant channels later. I’m also dedicating some time to monitoring bugs, mailing lists and questions on IRC, except when I want to focus and shut them off. I need some self-discipline to do less conversation and more action.

More regular entries to come!

Posted in GSoC | Tagged , | 1 Comment

Weekly Report #1: Diving In

Weekly report for the first two weeks of the Google Summer of Code

It’s On!

On May 24, GSoC officially began, and I started on my first task. The Python wiki page for Distutils2 lists my tasks under the “Distutils build tool” heading, i.e. “Distutils build command improvements”; the first one is adding a configure command to d2 (nickname for distutils2 on IRC, cause you know, we don’t have the time to type all these letters on the internets).

Learning Is Working Too

I did not manage to work more than 20-odd hours during the fisrt week. I did the first item off my list, studying 4Suite’s config command. Results of this are on my scratchpad.

To understand how it all works, I also read the build* and install* classes in Distutils2, and the base Command class. The picture starts to look quite clear in my mind. I refreshed some of my memories on docs.python.org/{install,distutils}, and acquired new knowledge by reading the Hitchhiker’s Guide to Packaging and the discussion threads on the associated google group. That will prove more useful in dealing with people (e.g. to switch a project to Distutils2) than to write code, but I think it was not wasted time. My mentor Titus agreed on that last point :)

Getting My Hands Dirty

During the second week, I’ve made progress with the configure command. I’ve learned a lot about the Command API and the relationships between commands, options, distribution and metadata. At the time of writing I’m nearly done with the basic command (with only two options for this first step) and I should be able to ask for review. I’ve read some test modules but not all of them; I’ll need to find how to use support code to write test that deal with file I/O, since my command writes and reads a file, or maybe write that support code. I expect this command to be ready by Monday 7 or 14.

While reading the code, I’ve made some small edits to fix py3k deprecation warnings and one bug. Tarek has merged them; you can see them on his repo or mine. I’ve also collected a number of typos and minor nitpicks; I’ll wait for the end of the summer for these unimportant things.


When I’m done with the configure command or waiting for feedback, I’ll start on killing setup.py scripts, first with #8252 (a metadata section for setup.cfg), then proceed to #8253 (describe Python files and other resources). I guess I’ll need a week or two.

I’m a bit disappointed by the results I have so far considering the time I’ve spent on front of my computer. I tend to be easily distracted and need some time to warm up and get coding. Perhaps I need to stay off IRC and email to focus. That said, it’s a great learning and coding experience, made even better by the nice people involved.

I’ve been wavering between two designs for the configure command, but putting it on paper helped me see the tradeoffs and choose one. Even if it’s shot down, at least it will be coherent :)

Posted in GSoC | Tagged , | Leave a comment

OpenForge Issues and Ideas

OpenForge 1.0 is a nice idea with a perfectible implementation. In this entry I’ll try to point out the problems I see and make suggestions for the next revision.

Reusing Prior Art

The XML format used by OpenForge seems to overlap with DOAP (description of a project). XML is supposed to promote reuse, so use FOAF to describe people, DOAP for projects, Trove classifiers for project metadata, Atom for news entries, OpenSearch for search results, etc. Inventing a new format (or an extension to DOAP) is OK for new things like repository metadata, but otherwise it’s only sensible to learn and reuse existing good ideas, which already have schematas, tests, libraries and tools.

Removing Unnecessary Restrictions

These paragraphs reference the sections named description and implementation.

Don’t mandate XML. JSON or YAML would be perfectly adequate here, and thanks to HTTP there is a simple way to get different representations for the same resource (the Accept header).

Don’t restrict project names. Or do it right, with a rationale for it, and a survey of restrictions imposed by common forge, hosting and cataloguing services.

Don’t recommend one URI space organization. Each forge may have good reasons for a particular layout. The key thing is to have a resource mapping defined resource (e.g. “project”) to a URI template (e.g. /p/{project}).

Furthermore, if you understand the spirit of REST, you’ll see that there is no need for an ugly /api namespace. A web browser can visit http://forge.example.org/p/something and get an HTML page about the project, and an indexing tool may use the same URI to get an XML file with the same content. HTTP is not RPC, it’s better, and it does not need /api.

Other Details

If you have to define an XML format, use an XML namespace, and version it. Example: http://openforge.info/ns/2.0#.

Use a formal language to define the format with enough precisions, both for humans and tools (generators, validators…). Use RelaxNG, it’s better than XML Schema or DTDs.

New Features

There’s one set of features missing from DOAP or OpenForge 1.0 that I’d like to see: VCS communication, or in other words, cross-forges hooks. At present, I can publish a repo for a project on a variety of sites, and someone may clone my repo, make changes that I can follow, and then send me a pull request. But this nice communication stops at the boundary between GitHub and CodingTeam, Bitbucket and Intuxication. I’d like OpenForge 2.0 to define how I cold easily clone a project from one site to another, follow updates in my service of choice, send pull requests to another site. Decentralized version control is honking great, having different hosting providers is flexible and healthy, and I think this missing glue would make them more useful.

Reaching Out

Probably the more difficult point, especially for such a small team: Get involved with other communities. Ask DOAP people if they’re willing to give feedback about your extensions and perhaps merge them in a future version. Ask VCS hosting services admins if they’re willing to implement OpenForge. Explain how OpenForge is a set of good practices that benefit users. Show pages at CodingTeam hosting that use the OpenForge features. Write a client that is able to query any OpenForge-enabled site. Publish screenshots.

Final Words

I think OpenForge is a project worth pushing, and I hope my few remarks are helpful. Kudos xbright for doing this!

Posted in Other | Tagged , , | Leave a comment

from gsoc import weblog

I’ll use this weblog to report my progress on my Distutils2 tasks during this summer. Stay tuned!

Posted in GSoC, Meta | Tagged , | Leave a comment