Skip to content
AtlanticBT Monogram
Article

Blocking Baidu or other user agents with junOS IDP

Often, a network administrator will need to block a troublesome user agent that’s causing load or security issues. If you are using junOS firewalls with an IDP subscription, you can craft a custom attack rule that blocks these users at the firewall itself.

Let’s craft a custom attack policy to block Baidu, a common troublesome web spider that doesn’t obey robots.txt rules.

Let’s begin by connecting to your border firewall using CLI. Enter the configuration mode and issue these commands:

set security idp custom-attack Baidu-Spider recommended-action close-client
set security idp custom-attack Baidu-Spider severity minor
set security idp custom-attack Baidu-Spider attack-type signature context http-header-user-agent
set security idp custom-attack Baidu-Spider attack-type signature pattern .*Baidu.*
set security idp custom-attack Baidu-Spider attack-type signature direction client-to-server

Next, add your newly crafted custom attack rule to your existing IDP policy that polices your incoming HTTP traffic.

You can modify the signaturepattern using regular expressions to block other user agents as well. If you are comfortable with junOS’s command line interface, this is an easy way to protect all of your webservers from this user agent. Your millage may vary, and you should follow Best Practices when making changes to your firewall configuration.

capabilities covered
Amazon Web Services

The Atlantic BT Manifesto

The Ultimate Guide To Planning A Complex Web Project

Insights

Atlantic BT's Insights

We’re sharing the latest concepts in tech, design, and software development. Learn more about our findings.

Questions & Answers

Are there differences in application architecture that are important for the cloud?
It is important to build applications and workloads specifically for the cloud. You will want to carefully consider what services the cloud provider of your choice has to offer and how your application leverages those services.
Learn More
Are there any drawbacks to cloud hosting?
Yes, there will always be some risks associated with any hosting option. You are relying on the resiliency and engineering of infrastructure that has scaled at an astounding rate.
Learn More
What’s the benefit of hosting in the cloud vs. traditional options?
Reasons not to host in the cloud are few and far between. If you don't host in the cloud, you will spend more in both CapEx and OpEx to manage your applications or websites in a traditional environment.
Learn More
How can I improve the performance of my application?
There are several primary reasons that applications perform poorly, and in some cases it’s a combination of several. 1) Data latency: If your application is making calls to a data source (whether it’s an API or a direct call) and there is latency at the data provider, your application performance will suffer.
Learn More
Should I move my application to the cloud?
The answer is ‘probably yes’. There aren’t many reasons for an application to be hosted elsewhere, aside from occasional compliance standards, or requirements to integrate with local services that would require large amounts of data to move from on-premise to cloud.
Learn More
Where should my application be hosted?
There are many different options for hosting, but most applications would do well with one of the cloud providers -- Amazon Web Services, Google Cloud Platform, Microsoft Azure.
Learn More