Taskcluster Github is the Taskcluster service responsible for kick starting tasks on Github repositories. At a high level: You install a Taskcluster app from the Github marketplace. This app sends webhooks to the Github service. Upon receiving a webhook, the Github service processes your repository’s .taskcluster.yml file. The Github service schedules tasks (if any) and updates the Github checks suite, or comments on your push / pull-request if there is an error. While the service itself is...| Blogs on Hunting the Shmoo
Last time we looked at some ways reviewers can keep the review process moving efficiently. This week, let’s put on our author hats and do the same thing.| Blogs on Hunting the Shmoo
In the next two posts we will examine the etiquette of using Phabricator. This post will examine tips from the reviewer’s perspective, and next week will focus on the author’s point of view. While the social aspects of etiquette are incredibly important, we should all be polite and considerate, these posts will focus more on the mechanics of using Phabricator. In other words, how to make the review process as smooth as possible without wasting anyone’s time. Let’s dig in!| Blogs on Hunting the Shmoo
If it ain’t broke don’t fix it. This old addage is valuable advice that has been passed down through generations. But it hasn’t stopped these people from rewriting command line tools perfected 30+ years ago in Rust. This week we’ll take a quick look at exa, a replacement for ls. So why should you ignore the wise advice from the addage and replace ls? Because there are marginal improvements to be had, duh! Although the improvements in this case are far from marginal.| Blogs on Hunting the Shmoo
Following up last week’s post on some mach try fundamentals, I figured it would be worth posting some actual concrete tips and tricks. So without further ado, here are some things you can do with ./mach try you may not have known about in rapid fire format.| Blogs on Hunting the Shmoo
There is a lot of confusion around mach try. People frequently ask “How do I get task X in mach try fuzzy?” or “How can I avoid getting backed out?”. This post is not so much a tip, rather an explanation around how mach try works and its relationship to the CI system (taskgraph). Armed with this knowledge, I hope you’ll be able to use mach try a little more effectively.| Blogs on Hunting the Shmoo
Mozilla developers often need to juggle multiple build configurations in their day to day work. Strategies to manage this sometimes include complex shell scripting built into their mozconfig, or a topsrcdir littered with mozconfig-* files and then calls to the build system like MOZCONFIG=mozconfig-debug ./mach build. But there’s another method (which is basically just a variant on the latter), that might help make managing mozconfigs a teensy bit easier: mozconfigwrapper. In the interest of...| Blogs on Hunting the Shmoo
Have you ever submitted a patch to Phabricator only to have reviewbot reveal dozens of lint errors all over? Or worse yet, have you landed before reviewbot had a chance to analyze your patch and been backed out over lint failures? If so fear not, we’ve all been there. Still, it’s hard not to feel a little embarrassed when it happens. Luckily for you, it’s pretty easy to eliminate the possibility of it ever happening again!| Blogs on Hunting the Shmoo
I first joined Mozilla as an intern in 2010 for the “Tools and Automation Team” (colloquially called the “A-Team”). I always had a bit of difficulty describing our role. We work on tests. But not the tests themselves, the the thing that runs the tests. Also we make sure the tests run when code lands. Also we have this dashboard to view results, oh and also we do a bunch of miscellaneous developer productivity kind of things. Oh and sometimes we have to do other operational type things...| Blogs on Hunting the Shmoo
If you’re working with mozilla-central on Windows and followed the official documentation, there’s a good chance the MozillaBuild shell is running in the default cmd.exe console. If you’ve spent any amount of time in this console you’ve also likely noticed it leaves a bit to be desired. Standard terminal features such as tabs, splits and themes are missing. More importantly, it doesn’t render unicode characters (at least out of the box). Luckily Microsoft has developed a modern term...| Blogs on Hunting the Shmoo
This post was co-authored with Marco Castelluccio, and was originally posted to the Mozilla Hacks Blog. A browser is an incredibly complex piece of software. With such enormous complexity, the only way to maintain a rapid pace of development is through an extensive CI system that can give developers confidence that their changes won’t introduce bugs. Given the scale of our CI, we’re always looking for ways to reduce load while maintaining a high standard of product quality. We wondered if...| Blogs on Hunting the Shmoo
Mozilla recently announced that we are planning to de-commission irc.mozilla.org in favour of a yet to be determined solution. As a long time user and supporter of IRC, this decision causes me some melancholy, but I 100% believe that it is the right call. Moreover, having had an inside glimpse at the process to replace it, I’m supremely confident whatever is chosen will be the best option for Mozilla’s needs. I’m not here to explain why deprecating IRC is a good idea. Otherpeople have a...| Blogs on Hunting the Shmoo
Mozilla uses a lot of Python. Most of our build system, CI configuration, test harnesses, command line tooling and countless other scripts, tools or Github projects are all handled by Python. In mozilla-central there are over 3500 Python files (excluding third party files), comprising roughly 230k lines of code. Additionally there are 462 repositories labelled with Python in the Mozilla org on Github (though many of these are not active). That’s a lot of Python, and most of it is Python 2. ...| Blogs on Hunting the Shmoo
I’ve recently redone my website and wanted to document a few of the changes and motivations behind them.| Blogs on Hunting the Shmoo
It’s no secret that I’m not a fan of try syntax, it’s a topic I’ve blogged about on severaloccasions before. Today, I’m pleased to announce that there’s a real alternative now landed on mozilla-central. It works on all platforms with mercurial and git. For those who just like to dive in: $ mach mercurial-setup --update # only if using hg$ mach try fuzzy This will prompt you to install fzf. After bootstrapping is finished, you’ll enter an interface populated with a list of all po...| Blogs on Hunting the Shmoo
Imagine this scenario. You’ve pushed a large series of commits to your favourite review tool (because you are a believer in the glory of microcommits). The reviewer however has found several problems, and worse, they are spread across all of the commits in your series. How do you fix all the issues with minimal fuss while preserving the commit order?| Blogs on Hunting the Shmoo
I’ve previously blogged about why I believe try syntax is an antiquated development process that should be replaced with something more modern and flexible. What follows is a series of ideas that I’m trying to convert into a concrete plan of action to bring this about. This is not an Intent to Implement or anything like that, but my hope is that this outline is detailed enough that it could be used as a solid starting point by someone with enough time and motivation to work on it.| Blogs on Hunting the Shmoo
One of the most painful aspects of a developer’s work cycle is trying to fix failures that show up on try, but which can’t be reproduced locally. When this happens, there were really only two options (neither of them nice): You could spam try with print debugging. But this isn’t very powerful, and takes forever to get feedback. You could request a loaner from releng. But this is a heavy handed process, and once you have the loaner it is very hard to get tests up and running. I’m pleas...| Blogs on Hunting the Shmoo
Mach is the Mozilla developer’s swiss army knife. It gathers all the important commands you’ll ever need to run, and puts them in one convenient place. Instead of hunting down documentation, or asking for help on irc, often a simple |mach help| is all that’s needed to get you started. Mach is great. But lately, mach is becoming more like the Mozilla developer’s toolbox. It still has everything you need but it weighs a ton, and it takes a good deal of rummaging around to find anything....| Blogs on Hunting the Shmoo
Today marks the 5 year anniversary of try syntax. For the uninitiated, try syntax is a string that you put into your commit message which a parser then uses to determine the set of builds and tests to run on your try push. A common try syntax might look like this: try: -b o -p linux -u mochitest -t none Since inception, it has been a core part of the Mozilla development workflow. For many years it has served us well, and even today it serves us passably. But it is almost time for try syntax t...| Blogs on Hunting the Shmoo
I mentioned in my previous post a mercurial extension I wrote for making bookmarks easier to manipulate. Since then it has undergone a large overhaul, and I believe it is now stable and intuitive enough to advertise a bit more widely. Introducing bookbinder When working with bookmarks (or anonymous heads) I often wanted to operate on the entire series of commits within the feature I was working on. I often found myself digging out revision numbers to find the first commit in a bookmark to do ...| Blogs on Hunting the Shmoo
This is a continuation of my previous post called The New Mercurial Workflow. It assumes that you have at least read and experimented with it a bit. If you haven’t, stop right now, read it, get set up and try playing around with bookmarks and mozreview a bit.| Blogs on Hunting the Shmoo
You may not know that most of our test harnesses are now outputting structured logs (thanks in large part to :chmanchester’s tireless work). Saying a log is structured simply means that it is in a machine readable format, in our case each log line is a JSON object. When streamed to a terminal or treeherder log, these JSON objects are first formatted into something that is human readable, aka the same log format you’re already familiar with (which is why you may not have noticed this).| Blogs on Hunting the Shmoo
There’s a good chance you’ve heard something about a new review tool coming to Mozilla and how it will change everything. There’s an even better chance you’ve stumbled across one of gps’ blog posts on how we use mercurial at Mozilla. With mozreview entering beta, I decided to throw out my old mq based workflow and try to use all the latest and greatest tools. That means mercurial bookmarks, a unified mozilla-central, using mozreview and completely expunging mq from my workflow.| Blogs on Hunting the Shmoo
tl;dr Look for reports like this in the near future! At Mozilla, platform developers are culturally bound to tbpl. We spend a lot of time staring at those bright little letters, and their colour can mean the difference between hours, days or even weeks of work. With so many people performing over 420 pushes per day, all watching, praying, rejoicing and cursing, it’s paramount that the whole process operates like a well oiled machine.| Blogs on Hunting the Shmoo
That’s not a rhetorical question. I’d like to know in which scenarios a mixin in python really is the best option. I can’t seem to think of any, but maybe I’m not thinking outside the box enough. The basic idea of a mixin is to create a small re-usable class that can “plug-in” to other larger classes. From the wikipedia definition, a mixin is a way to compose classes together without using inheritance. The problem is unlike ruby, python mixins are a purely conceptual construct. Py...| Blogs on Hunting the Shmoo
My last post was basically a very long winded way of saying, “we have a problem”. It kind of did a little dance around “why is there a problem” and “how do we fix it”, but I want to explore these two questions in a bit more detail. Specifically, I want to return to the two case studies and explore why our test harnesses don’t work and why mozharness does work even though both have IFFY (in flux for years) requirements. Then I will explore how to use the lessons learned to improv...| Blogs on Hunting the Shmoo
Dry versus Wet As programmers, we are taught early on that code duplication is bad and should be avoided at all cost. It makes code less maintainable, reusable and readable. The DRY principle is very basic and fundamental to how most of us approach software design. If you aren’t familiar with the DRY principle, please take a minute to read the wikipedia page on it. The counterpart of DRY, is WET (write everything twice). In general, I agree that DRY is good and WET is bad. But I think there...| Blogs on Hunting the Shmoo
Getting Started tl;dr - It is possible to add more mach to your B2G repo! To get started, install pip: $ wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py -O - | python Install b2g-commands: $ pip install b2g-commands To play around with it, cd to your B2G repo and run: $ git pull # make sure repo is up to date$ ./mach help # see all available commands$ ./mach help <command> # see additional info about a command Details Most people who spend the majority of their time worki...| Blogs on Hunting the Shmoo
Mach is an amazing tool which facilitates a large number of common user stories in the mozilla source tree. You can perform initial setup, execute a build, run tests, examine diagnostics, even search Google. Many of these things require an object directory. This can potentially lead to some confusion if you typically have more than one object directory at any given time. How does mach know which object directory to operate on?| Blogs on Hunting the Shmoo
This probably isn’t a groundbreaking revelation to most Mozillians, but it’s something I haven’t quite managed to put into words until now. On the Tools and Automation team at Mozilla, the easiest way to describe what we do is to say “We try to make the lives of others within the community easier”. This is a vague statement, but it’s about as close as we can get without writing a small dissertation.| Blogs on Hunting the Shmoo
Before now running ‘classic’ unittests (mochitest, reftest, xpcshell, etc.) on B2G emulators has been a massive pain. The new recommended way of running them is through mach. Update B2G repo if you haven’t already: git pull Configure an emulator: BRANCH=master ./config.sh emulator Build: ./build.sh Run: ./mach mochitest-remote| Blogs on Hunting the Shmoo
OUTDATED The steps in this article are no longer accurate. Pushing a custom command line is easier now. See here for more details. --- Have you ever wanted to see the test results of a custom command line in try? Things like –test-manifest, –shuffle or –run-slower? Now you can! The process isn’t exactly optimized for the developer use case, but neither is it really difficult to do once you know how.| Blogs on Hunting the Shmoo
qqver is a Mercurial extension that forces mq to version all patch queues in the same repository. The syntax is the exact same as when doing normal patch queue versioning (i.e ‘hg init –mq’), except the repo is created one directory higher so it is able to track all patch queues. As with stock queue repos, new and existing patches are added automatically.| Blogs on Hunting the Shmoo
I came across an odd python problem the other day. I wanted to format a log message without printing it with python’s logging module. But I couldn’t find any examples or others who wanted to do the same thing. You might ask why I would want to do this. In my case, I was using sys.stdout.write() to display the progress of a download as a percentage. Python’s logging module automatically appends a newline to your message and there didn’t seem to be a way to change this without a subclas...| Blogs on Hunting the Shmoo
Hey! Check it out, I have crash stacks in my logs now! I know, I know, I took my sweet time. I’m sorry. I hope this doesn’t damage our relationship. Is there anything I can do to make it up to you? I want to help. I really do. It’s just. I don’t know. I guess I’ve been feeling under the weather lately. I must have caught a bug.| Blogs on Hunting the Shmoo
Throughout most of our B2G test automation deployment, we’ve been very conscious about not enabling too many tests simply because we didn’t have enough capacity on our test slaves to run them all. Regardless it was still bad enough as it was (many of you probably experienced very long wait times for results). Thanks to releng (and especially Rail Aliiev) we are now running most of our B2G tests in Amazon AWS which means we can be much more flexible in accomodating load.| Blogs on Hunting the Shmoo
As B2G continues to trod onwards to its release, there is still a lot of confusion about the level and state of test coverage it has. Back in November we started running mochitests, reftests and marionette/webapi tests on ARM emulators. Now we’ve also added xpcshell tests and for the most part we have these nice green letters to look at on TBPL that make us feel good about ourselves. But what is really being run? What is the meaning behind these letters “M”, “R”, “Mn” and “X...| Blogs on Hunting the Shmoo
Contrary to popular belief, we (the A-Team) have been running mochitests, reftests, marionette tests and webapi tests on B2G in some form of continuous integration or another for about 5 months now. They’ve been reporting results to a TBPL look-alike called autolog, and were run on Amazon EC2 VM’s with emulators. This was a temporary solution to get something stood up quickly while we moved towards our ultimate B2G automation goal - tests running on Pandaboards and reporting to TBPL. As o...| Blogs on Hunting the Shmoo
This quarter I’ve been focusing on getting reftests running on B2G, triaging them and fixing various issues. The purpose of this post is to outline their status, go over the work that still needs to be done and point out where I will need some help.| Blogs on Hunting the Shmoo
Sometimes I’d remember this awesome thing that I saw on Reddit awhile back and want to share it with someone. I’d try to find it through browser history or reddit search, but I wasn’t usually successful. Reddit search isn’t very good and I use Reddit a lot so my browser history for reddit.com is huge and unwieldy. So in my spare time I decided to write Reddit History. It basically just keeps track of all the submissions you’ve viewed, and lets you filter them by sub-reddit and text ...| Blogs on Hunting the Shmoo
Normally I don’t bother making top ten lists at the end of the year, but I was bored and figured, why not? So without further ado, here are my top ten albums of 2011. Unexpect - Fables of the Sleepless Empire The Decemberists - The King is Dead Battles - Gloss Drop St. Vincent - Strange Mercy The Dear Hunter - The Color Spectrum Fucked Up - David Comes to Life Cut Copy - Zonoscope Panda Bear - Tomboy Phideaux - Snowtorch James Blake - James Blake| Blogs on Hunting the Shmoo
Mozconfigwrapper is a tool inspired by Doug Hellman’s magnificent virtualenvwrapper. In a nutshell, mozconfigwrapper hides all of your mozconfigs into a configurable directory (defaults to ~/.mozconfigs), and lets you easily switch, create, remove, edit and list them. Mozconfigwrapper is Unix only for now.| Blogs on Hunting the Shmoo
So here in Toronto another cyclist was killed today. I’m a little bit frustrated with the state of cycling in Toronto in general. On one hand you have people like Mr. Ford who think that cyclists are always to blame and should be run off the road, and on the other you have cyclists who actually do run reds and wear black in the middle of the night giving the rest of us a bad name. I decided I need to write down my thoughts if for no other reason than to calm down. In an ideal world, cyclist...| Blogs on Hunting the Shmoo
While responsiveness is one of the main goals for Firefox this quarter, we still don’t quite have the means to measure and test our progress towards this goal. The good news is that there are, and have been for some time, several efforts to fix this problem. Back in June, Ted wrote some event tracing instrumentation that gives us a reasonable idea of when the browser becomes unresponsive. This event tracer is already being used by some Talos tests which gives us a good general idea of wheth...| Blogs on Hunting the Shmoo
At the beginning of September, I was asked to write yet another automated test harness for testing user responsiveness. Among other things, the harness needed to be capable of automating a wide range of user interactions in Firefox (such as opening context menus, clicking buttons etc). Oh and by the way this needs to be finished as quickly as possible.| Blogs on Hunting the Shmoo
While developing an Android app for my final design project, Taedium, I came across what seemed like a trivial problem. I needed to have a text view (like that in the Android market) that can expand or collapse whenever a user presses a ‘More’ or ‘Less’ button. It turned out that the problem was more difficult than I thought and I had to make my own custom widget to accomplish the task. I based it off a partial solution by Peteris Caune over on Stack Overflow.| Blogs on Hunting the Shmoo
Before I started interning at Mozilla back in May 2010, I really didn’t know what to expect. How does a non-profit company with an open source product operate? After working at giant corporations like IBM and McAfee I couldn’t fathom what the experience would be like.| Blogs on Hunting the Shmoo
Firefox is known for its extensibility. In fact, over 2.4 billion addons have been downloaded to date, meaning there are a lot of people using a lot of addons. While having 20+ addons can undoubtedly personalize your browsing experience, it can also be a pain in the ass to manually install them every time you set up a new Firefox profile. As a developer working on Firefox related automation tools, this is twice as true since I create a separate profile for each and every project I work on, in...| Blogs on Hunting the Shmoo
One of the pleasures of working at Mozilla, has been learning and using the Mercurial version control system. Over the past decade, I’ve spent countless hours tinkering my worfklow to be just so. Reading docs and articles, meticulously tweaking settings and even writing an extension. I used to be very passionate about Mercurial. But as time went on, the culture at Mozilla started changing. More and more repos were created in Github, and more and more developers started using git-cinnabar to...| ahal.ca