What is the Google Chrome Headless Mode?
All you Need to Know about Headless Chrome
The Headless mode is a feature which allows the execution of a full version of the Chrome Browser. It provides the ability to control Chrome via external programs. The headless mode can run on servers without the need for dedicated display or graphics. It gets the name headless because it runs without the Graphical User Interface (GUI). The GUI of a program is referred to as the head.
The headless mode of the Google Chrome browser provides the ability to implement and run massive-scale web app tests, navigate from one page to another without human interference, validate JavaScript functions, and provide detailed reporting. In short, it is a step towards automation.
This article will discuss the headless mode that Google added to Chrome in 2017 and has subsequently become extremely popular among software engineers, testers, and even attackers and how it bypassed Selenium and other popular softwares.
Google Headless Chrome is not the first entry into the automation paradigm. Automation has long been used in other headless browsers, including the PhantomJS and Nightmare JS.
PhantomJS is a discontinued headless browser used for automating web page interaction whereas the Nightmare JS is a browser automation library for node.js.
Test frameworks are also automated, such as the Capybara and Jasmin. Both of these softwares simulate scenarios for user stories and automate the web application testing based upon behavior-driven development.
Selenium is a top-rated app that automates many browsers, including Google Chrome web browser. Selenium provides the option for a browser add on that can easily replicate the user interactions and workflows.
The popularity of Headless Chrome
Google Headless Chrome has been extremely popular since its launch in 2017. Within a year, Google Chrome Browser took over the previous automation leader PhantomJS. Cloud WAF statistics following the Google trends clearly show the popularity of Chrome Headless browser increasing while the PhantomJS loses its earlier status.
The following chart confirms this trend.
How Automated Browsers have increased the online traffic?
An automated web browser generates much more Traffic compared to non-automated web surfing. This is because Google Headless Chrome is continuously increasing in popularity and taking over the manual paradigm while the PhantomJS, Selenium, etc. are slowly becoming irrelevant.
The diagram below verifies our claim of the increased popularity of automated browsers.
What makes Google Headless Chrome Browser Popular?
Several factors contribute to the success of Google Headless Chrome Browser.
- The biggest one is Chrome’s support for innovative features that continuously provide out-of-the-box innovation and web development trends.
- Headless Chrome offers user-friendly tools for web development and additional helpful features for the developers.
- Another feature is the availability of this Headless Chrome on all operating systems including Windows, Mac, and Linux.
Google released Puppeteer a few months after the Chrome headless functionality was unveiled. This tactic boosted the popularity of Headless Chrome amongst developers and software engineers since they now had more options. The popularity of Selenium took a great hit once it was unveiled.
What is the Puppeteer API?
Puppeteer is a node library feature which was launched a few months after the headless feature was introduced. The Puppeteer node library is user-friendly and provides flexibility to manipulate many Chrome functions.
You get full browser functionality with Puppeteer in addition to running Chrome in headless mode on a server. Team managing the automated paradigm greatly benefits from the remote server capability.
The Puppeteer Node API will not provide you with all the standard functions of Google Chrome browser. However, you will get the flexibility to write the automation code in any programming language you are comfortable with, excluding NodeJS and JavaScript.
The latest version of the Puppeteer Node API can fortify the functionality of Headless Chrome.
Learn more about Puppeteer API from the GitHub URL https://github.com/puppeteer/puppeteer. You can find a step by step process and learn how to use it and set it up.
All code along with the set up is available on GitHub.
Puppeteer API is not the only solution to automate Chrome. Other options are also available. Webdriver or third-party frameworks such as Selenium or direct Command Line Interface (CLI) can also be used for this purpose.
How to Setup the Chrome Headless Browser
To use Headless Chrome, you will need to open the binary Chrome from the command line. For an average user, the commands and the interface might be a little overwhelming, but we will provide a detailed step-by-step process.
- Start by opening the command line on your Operating System. It is already present in operating systems. No need to install an external driver or import any library. It also comes integrated with Linux.
MAC Users: Applications -> Utilities -> Terminal
WINDOWS USERS: Open Start and enter cmd in the search bar. Click the icon to open the Command-Line.
- Once the command line is open, we will open Chrome using it. You need to know where Chrome is stored on your machine. The location should not be confused for an icon. Depending upon your system copy and paste the following driver commands:
MAC USERS: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
WINDOWS USERS: C:Program FilesGoogleChromeApplication
- You will require a separate version of Chrome that runs as an independent application. You will need Google Chrome Canary. Users can download the latest version of the driver from this URL https://www.google.com/intl/en_pk/chrome/canary/. Once Google Chrome Canary is downloaded, we will open it in the command line. The set up is quite simple and takes a few minutes. Mac users can copy and paste the following driver command:
MAC USERS: /Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary
Canary will open up in a new window. Windows users will have to amend their file path to reach Canary place inside the C drive.
- To make Google Chrome Browser Headless, we will use the same command as we used before, but we will add this time -headless with it. So the driver command becomes:
/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary –headless
The Windows Users will do the same thing with the Command-Line.
Windows USERS:
System.setProperty(“webdriver.chrome.driver”, “/path/to/driver”);
ChromeOptions options = new ChromeOptions();
options.addArguments(“–headless”);
WebDriver driver = new ChromeDriver (options);
- The Yellow Chrome Canary symbol will appear in the document for a split second and then disappear. It ensures that Chrome Canary runs in Headless mode.
Activities Performed Using the Headless Chrome Browser
The advantages offered by the Headless Chrome software are tremendous. Let us list down a few options that you can perform using Google headless Chrome. Some of these cool features are unique to this software and were not available on previous tools such as Selenium, etc.
Dynamic Rendering
The search engines utilize Headless Chrome to render the page, display runtime contents, and index the material for single-page web apps.
Page Ranking
The search engine optimization tools offered by Headless Chrome provide a robust analysis for your web application and identify the weak areas. You can improve the shortcomings to set up higher rankings.
Runtime Calculation
Headless Chrome provides tools that help evaluate the performance of JavaScript runtime execution. It can be useful to optimize the code to ensure faster speeds. It is faster than other available options.
Improved Testing
Several online tools help us test the page rendering and compare it with the antecedent version to look for any changes or difficulties that might affect the user experience. An automated test can ensure the correct functioning of the latest version.
The list does not just end here. The capabilities are endless.
Cool Options to Play Around With Headless Chrome Browser
Since we have learned how to launch and manipulate headless chrome using the command line and Canary, let us dig deeper into the capabilities with this software. All these features can be replicated on a Linux as well.
Launching a Web Page
Learning how to launch the browser in headless mode will not be of much value. You will be required to open a web page to utilise all the features.
Visiting a web page is relatively easy. Add the desired URL at the end of the headless flag headers in the command line interface.
Following is an example for Mac users. Use the following driver command:
MAC USERS: /Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary –headless https://www.google.com
Note: You can add any URL at the end.
As discussed before. The Canary icon will show up and instantly disappear in the Doc. You will not see more than this. To delve deeper into the specifics, take a screenshot of the page in the command line.
Take a Screenshot
You can take a screenshot with the help of headers. Use the following example driver command:
/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary –headless –disable-GPU –screenshot https://www.google.com
Note: You can add any URL at the end.
You will get a notification on the command line with the destination info. By default, it will be named screenshot.png.
For Mac Users, it will be stored in the Home folder but will overwrite the previous screenshot.png file. Make sure to save it.
You will only get the image of one page. To capture the complete web page, you will need a PDF of the web page.
Add the print to pdf header at the end of the command line. Use the following example driver command:
/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary –headless –disable-gpu –print-to-pdf www.google.com
The result will be saved as an output.pdf file in the home folder of Mac. Make sure to rename this file. Otherwise, it will be overwritten every time you make a new copy.
Manipulating DevTools from the Command Line
Headless Chrome can be used to create a remote instance of the Chrome DevTools. This feature is convenient when it comes to controlling the Headless Chrome.
Make sure to find out your port number before trying this out.
Like the previous commands, we will achieve this feat if we add headers at the end.
–remote-debugging-port=8080
Any port can be set, but it is a good practice to add the default one if you lack the technical experience. Use the following example driver command:
/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary –headless –remote-debugging-port=8080 www.google.com
Note: You can add any URL at the end.
- Close and reopen Canary when you use this command.
- You will get a notification from the chrome devtools when they are ready.
- Copy the URL that you get and paste it in a new tab on the traditional Google Chrome.
- Make sure to keep the port numbers consistent.
Downside to Headless Chrome
Looking at the previous data, it is easy to decipher that the earlier leader of automated browsing, i.e PhantomJS was a top-notch traffic magnet. However, it was a leader in malicious activity as well.
Ever since Chrome browser has taken over the automated paradigm, it also occupies the first spot in the attacker’s podium. The vitriolic Traffic is pretty evenly divided between the execution in headless as well as the non-headless mode.
However, the trends reveal that there is no such preference amongst the hackers to choose Headless Chrome for carrying out a malevolent activity such as exploiting loopholes, database corruption using SQL injection, or a cross-site scripting attack (XSS) using a driver.
Web browsers are popularly used for this purpose because they can easily bypass the web validation techniques. This technique requires skill but is not new. Even Selenium based tools can accomplish this feat.
Why is a Headless Browser not used for DDoS Attacks?
If we analyze the activity trends from 2017, we can see no DDoS attack performed using a Headless Chrome or the Headless Android Botnet.
Using automated browsers such as Headless Chrome for undertaking a DDoS attack is not one of the widespread options. It is because the browsers generate a low request rate to the server. Upon receiving the server’s response, Chrome first evaluates the contents and then moves on to the next request. The sequential checking of packets makes the entire process useless.
Instead, hackers resort to a single piece of the script having the potential to flood the server with more requests than it can handle. There is no wait for the response, potentially choking the server. A disaster awaits the website once the server goes down.
This does not rule out that headless Chrome is not utilized to carry out other sorts of malicious activity. More than ten thousand IP addresses perform scraping, sniping, blackhat Search Engine Optimization, etc., and different kinds of activity using JavaScript evaluation on the headless Chrome. Many attackers use a Virtual Private Network or utilize other encapsulating options to conceal their identity. It can prove to be more fatal than Selenium if misused.
Difference Between Chrome and Chromium
| Chrome | Chromium | |
| Features | Better update mechanism, native support for many technologies, rights to play copyrighted content | Lacks all of these features |
| Stability | Extremely stable | Can crash quite often even in comparison with primitive versions of Chrome |
| Privacy | Collects detailed information including usage patterns, statistics, crash reports, etc. and sends it to Google. | Better for privacy. Can not collect and send information. |
| Security | Security is similar to Chromium since it is powered by it. Chrome automatically updates and applies all the security patches. | Same security as Chrome but lack of automated update mechanism. |
| License Support | All media formats are supported natively. | Only free and basic codecs are supported. |
Which command starts the google chrome web browser in headless mode?
As we have already seen, you just have to add the flag –headless when you launch the browser to be in headless mode.
With CLI (Command Line Interface), just write:
chrome \<br> --headless \ # Runs Chrome in headless mode.<br> --disable-gpu \ # Temporarily needed if running on Windows.<br> --remote-debugging-port=9222 \<br> https://chromestatus.com/roadmap # URL to open. Defaults to about:blank.If you want to print the DOM (Document Object Model), just write the flag –dump-dom. It will write the document.body.innerHTML to stdout:
chrome --headless --disable-gpu --dump-dom https://chromestatus.com/roadmapMore information here: https://developer.chrome.com/blog/headless-chrome/
Conclusion
Google Headless Chrome browser in regular usage is not malicious. It is much more powerful than previous tools including Selenium.
We learnt how the combination of Puppeteer Node API and Headless Chrome can prove to be a game changer along with the GitHub URL for a detailed overview. We have discussed many scenarios when this feature can add actual value to your workflows.
Initially, it might become a little challenging to play around with this software, but once you get the hang of it, the options are endless.
We also discussed various commands and their driver to understand how to manipulate options.
The Chrome Headless Browser is surely a better option for an average user compared to Selenium with the ability to run on Linux
