So you have booted up your vulnerable VM, found the IP address and run NMap to find the open services and ports. You have found a web server running on port 80 (http) or 443 (https). You load the IP address or domain in your browser but all you find are some default webserver pages or some simple web pages with no links.
At this point it is always worth right-clicking to view the HTML source. Sometimes comments or hints are contained within the code.
If you cannot find anything on the page or in the code it is time to try and search for hidden webpages. This can be done by using a command line tool such as DIRB. DIRB queries the web server asking if a page exists. It does this using a list of common directory or web page names.
To use dirb from the command line run the following command:
dirb http://172.16.195.202 /usr/share/wordlist/dirb/common.txt
Replace the IP address with IP of the target machine. Ensure that the http:// or https:// part of the address is present. https:// would be used if the webserver is running over ssh on port 443.
The second part of the launch command is the route to the directory containing the dictionary of common web page names. This is true only for Kali. If you aren’t using kali your wordlist maybe located elsewhere. You need to find your wordlist and add that path.
In Kali go to the applications menu, web application analysis and click the down arrow. Select the web crawlers and directory brute force menu and then select DIRBuster.
In the window that loads:
Enter the IP address of the target machine taking care to enter the http for port 80 or https if using SSL over port 443.
If scanning a local machine select the GoFaster checkbox.
Browse to and select the wordlist on your machine you intend to use. There is nothing stopping you running the scan again and again each time using a different wordlist.
Under file extension add the file extensions you would like to look at. The default is php but if you have determined the target site is using static html files then it will be worth adding .html.