Behat is a BDD framework for PHP. It uses a Gherkin-like structure to write human-readable tests, like these:
Scenario: Finding some cheese Given I am on the Google search page When I search for "Cheese!" Then the page title should start with "cheese"
(A bunch of scenarios are put in a .feature file.)
While you can express tests for anything (for example, the output of a program or function), it is often used in the context of automated web application testing: write a test suite that goes through the flow of your web application (say, a simple webshop), and if all the tests are green, that means you’re in the clear! (Or you just haven’t written enough tests yet.)
Obviously, such a thing would be very nice to have for Haskell!
A rough battle plan was written out here, and some related repositories are mentioned.
Unfortunately, the majority of these packages are all very old (>3 years), and will not have been kept up to spec with the Gherkin grammar.
On top of that, there are many elements that allow us to write a few tests and have them executed automatically on a (headless) browser, which the battle plan does not cover.
Here’s how Behat does it:
Now let’s look at each element in detail, but starting from the bottom.
This library establishes a connection with a Selenium2 server, opens a session, and sends commands to it. These are all just curl calls.
Do we have a Haskell equivalent? Most certainly! The webdriver package is well-maintained, and seems to support everything.
Mink defines a base class for a web driver and a bunch of utility functions, such as converters from a CSS class selector (“Click the element with CSS class ‘submit’”), a named selector (“Click the button labeled ‘Start’”), etc., to an XPath selector, which most web drivers understand.
It also provides a few implementations of web drivers to fit into its system. Most of these implementations also expose the driver-specific capabilities or functionality.
Do we have a Haskell equivalent? No. Such a library would provide an interface for arbitrary web drivers. Mink, by itself, does not handle the translation of Behat steps to web driver commands. The Mink extension does that.
In my opinion, this abstraction layer is not required to get something Behatty running for Haskell. We can add it later and just have Selenium2 as the only concrete web driver for now.
This provides a few Contexts that allow Behat to write steps that should send commands to a web driver.
For instance, the RawMinkContext has a function for visiting a page:
public function visitPath($path, $sessionName = null)
{
$this->getSession($sessionName)->visit($this->locatePath($path));
}getSession being defined as:
public function getSession($name = null)
{
return $this->getMink()->getSession($name);
}getMink() fetches the web driver instance, and getSession() fetches the active web driver session.
(And visit(), finally, takes a relative path and translates that into a command for the web driver.)
With these kinds of building blocks, you can then write your own custom steps, which you can then refer to in your scenario.
Do we have a Haskell equivalent? No. And seeing as we’ll probably skip Mink for the moment, it’s not yet required.
Apart from parsing files and running the functions that map to each parsed step, Behat also handles configuration of the web driver (Which browser? Which version? Javascript on or not?) via the behat.yml file.
Do we have a Haskell equivalent? Kinda.
For the parser part, we have the abacate package, but it is very old, and the BNF grammar to parse the Gherkin language is probably out of date.
The Gherkin language is essentially the language to write those human-readable scenarios we mentioned earlier. It’s managed by the Cucumber team, which is something like Behat.
The Gherkin language grammar itself is written in Berp, which is a cross-language parser generator.
Here’s how Cucumber describes its workings:
Berp takes a grammar file (
gherkin.berp) and a template file (gherkin-X.razor) as input and outputs a parser in language X:╔════════════╗ ┌────────┐ ╔═══════════════╗ ║gherkin.berp║──>│berp.exe│<──║gherkin-X.razor║ ╚════════════╝ └────────┘ ╚═══════════════╝ │ V ╔════════╗ ║Parser.x║ ╚════════╝
So, essentially, you would put the Haskell code to generate a Haskell-based parser in the gherkin-X.razor file, and out would come Parser.hs. Then, Parser.hs would be used in the rest of the Cucumber workflow:
The following diagram outlines the architecture:
╔════════════╗ ┌───────┐ ╔══════╗ ┌──────┐ ╔═══╗ ║Feature file║──>│Scanner│──>║Tokens║──>│Parser│──>║AST║ ╚════════════╝ └───────┘ ╚══════╝ └──────┘ ╚═══╝The scanner reads a gherkin doc (typically read from a
.featurefile) and creates a token for each line. The tokens are passed to the parser, which outputs an AST (Abstract Syntax Tree).
So, Parser.hs would have to output an AST (in JSON.).
Finally, we need to “compile” the AST to a “pickle”:
The AST isn’t suitable for execution by Cucumber. It needs further processing into a simpler form called Pickles.
The compiler compiles the AST produced by the parser into pickles:
╔═══╗ ┌────────┐ ╔═══════╗ ║AST║──>│Compiler│──>║Pickles║ ╚═══╝ └────────┘ ╚═══════╝
(This also outputs JSON.)
When that’s all done, you would have a Cucumber-compliant implementation.
Of course, Haskell is quite good in parsing. Generating a parser (which would have to consume a bunch of tokens) and then writing a simple compiler to simplify the AST should be quite easy.
However, it seems easier to have a 100% Haskell package first that can produce an executable from a bunch of steps, which is able to:
With this proof-of-concept package, full Cucumber compliance could be attained at a later stage.
As for the non-parsing part, chuchu appears to have some of the step definition functionality available.
Ok, so what to do?
A good first step seems to be to update abacate. A limitation compared to a Cucumber implementation is that it would only be able to parse English feature files, but it’s a good start, and we could probably reuse it in the Cucumber-compliant version.
Then, we need to a way to collect the different step definitions. The chuchu library uses a ChuChu monad that is able to combine different steps via (>>=) and then just goes through the steps with (<|>). It don’t know how viable this is when collecting step definitions from different files or libraries, though.
When we have an updated version abacate and something like chuchu that allows us to take feature files and interpret them, we can write a base library consisting of step defintions that interface with webdriver.
After that, we should put it all to the test. :)