Gravityscan Blog

Updates on website security and what's going on in our universe.

Gravityscan’s First Day Results: In a Word – Wow!

This entry was posted in News on May 17, 2017 by mark 53 Replies

You only realize how incredibly impressive a team is on launch day. The Gravityscan team worked steadily for almost a year, consistently producing releases that added features as Gravityscan grew and became a product. Then, through the QA cycle, the team steadily burned down bugs and made the product rock-solid and ready for launch.

Here Are the Numbers

In the first 24 hours since Gravityscan launched, we processed 26,153 scans.

12,596 unique sites have been added to users’ accounts.

Of those, 6,007 sites had their site ownership verified with Google Analytics, which is by far the fastest and easiest method to verify site ownership. Remember: you need to verify site ownership to see vulnerabilities. We do this to make sure unauthorized users can’t see your site’s vulnerabilities.

We already have our first Pro customers, and many have upgraded multiple sites – in some cases, those upgrades numbered in the double digits – to Gravityscan Pro for faster scans and all the other benefits of Pro.

We have a total of 4,052 registered users now – and climbing.

The Craziness of Launch Day

Yesterday morning starting at 7am Pacific Time, we launched. We let our customers know in groups of about 20,000. Then we upped that to groups of 100,000. Then, finally, groups of 150,000.

Traffic on Gravityscan steadily increased, and everything held together. Until…

When we hit 900 concurrent scans, we started seeing some unexpected behavior. Some of our scan workers had stopped accepting jobs. The team got together and had an emergency meeting. We had an idea about what might be happening, but we didn’t have time to roll out a fix. So we chatted about options.

Our team had the foresight to build resilience in to Gravityscan, even when under extreme load. So what we decided to do was simply kill the scan workers that were no longer accepting jobs and restart them.

We made the call months ago to build Gravityscan for scale and make it incredibly resilient, and the team really delivered. They had designed the application so that if, for some reason, scan workers die, the scan jobs are cancelled and gracefully resubmitted, and our customers receive their results without a hitch.

We started new scan workers and killed the old ones, and everything worked exactly as designed. All affected scan jobs were resubmitted and everything worked perfectly.

And the customer experience? Only if you were paying close attention would you have noticed that your scan progress bar reset to zero and then went ahead and ran the full scan without a hitch. Fortunately, too, this only affected a handful of the large number of customers we had on Gravityscan.com yesterday.

The Benefits of a Trial by Fire

The truth is, we could have spread the launch over days, but we chose not to. Gravityscan is built for scale. We want to be the best vulnerability and malware scan for websites on the Internet. We also want to provide most of the features and benefits of Gravityscan to the online community for free.

That means we must be able to scale rapidly, and we need to be able to do that from launch day. So on launch we made the conscious choice to rapidly accelerate site traffic while closely monitoring performance. This provided us with an opportunity to see exactly how things perform under load and to make Gravityscan even better.

And This Is What We Fixed or Improved in Just 24 Hours

The team has moved incredibly quickly to find the underlying cause of the hiccup we had yesterday and they have gone way beyond that to respond quickly to bug reports.

I’d like to thank all of you, the brand new Gravityscan community, for the bug reports that you submitted via our contact form. The team has been organizing, prioritizing and rapidly fixing the issues you submitted. Your feedback is incredibly helpful and provides us what we need to rapidly improve the product.

Here are some of the fixes we’ve implemented and that are in production since yesterday:

  • Fixed the underlying issue that caused scan workers to stop accepting jobs. Everything is super fast and stable now, even at extreme load.
  • Removed JustSpam from the blacklists we check when evaluating your site reputation. Their removal process isn’t clear, so they’re out.
  • We had a scan signature throwing false positives for malware. We removed it from production.
  • We further improved real-time system monitoring so that if we have any load issue in future we are immediately alerted. We also increased the items we are monitoring based on yesterday’s experience.
  • Increased hardware available to scan workers to provide even more headroom, over and above the fix implemented.
  • Added a third physical server for scanning to further increase capacity.
  • Tightened up WordPress version-detection.
  • Changed text on certain result types to make it clearer what problem we detected.
  • Improved text description for Cloudflare customers to make it clear we know you’re running Cloudflare and to provide suggested config improvements to enhance scan reliability.
  • Added detection for the new Joomla vulnerability that emerged today.
  • Added detection for the WordPress vulnerabilities that were announced yesterday and fixed in the newest version.
  • Added support for usernames with apostrophes.
  • Made Google Analytics site verification messages more descriptive and helpful. It is now clear what the error is if you encounter a problem verifying your site with GA.
  • We now support some of the newer gTLDs, so if your site uses a mysite.technology or some other long TLD, you will no longer encounter validation errors when adding your site.
  • Fixed a formatting error when displaying errors to users.
  • Increased our ability to queue jobs in a predictable and stable manner when under high load.

Those are the highlights. We fixed several more minor issues, and the team continues to move at a rapid pace today to add requested functionality to Gravityscan.

Get the Word Out!

We need to let website owners know that there is now a free way to get a thorough malware and vulnerability scan for your website. You no longer need to pay certain enterprise companies thousands of dollars per year for a vulnerability scanner for your website. It’s free!

So I’d like to ask you to use the share buttons below to post Gravityscan to Facebook and Twitter and help us take back the Web from hackers!

 

Tagged with

Did you enjoy this post? Share it!


53 Replies on "Gravityscan’s First Day Results: In a Word – Wow!"

Michael May 29, 2017 at 5:50 pmReply

Holy cow. We scanned three of our websites and found some very shady malware hiding on 2 of our wordpress sites and 1 prestashop site.

Thank you guys for your services.

Arie Klerk May 21, 2017 at 1:36 pmReply

Great work guys! Though all my sites are OK, I realize that this program will help me prevent getting in trouble.
I have two problems:
1.: I wanted to subscribe to the newsletter, but received the following response:
Confirm Your Subscription
You have not yet confirmed your subscription.
To confirm it, follow the instructions in the email sent to you.
If you do not see that message in your inbox, check your bulk folder.

I do not see any confirmation email, do not know what to look for (what's the sender's address?) and do not know what my bulk folder is (it is not in spam...). Neither can I ask for a retransmission...

2.: I do not understand what to do with the following Low Severity Result:
HTTP Security Header information:
Content-Security-Policy: CSP not found on this site
- Cookies: Cookies not set on this site
- Public-Key-Pins: HPKP not applicable (requires https)
- Strict-Transport-Security: HSTS not applicable (requires https)
- X-Content-Type-Options: X-Content-Type-Options is set on this site
- X-Frame-Options: X-Frame-Options is set to sameorigin or deny
- X-XSS-Protection: X-XSS-Protection is enabled with mode block

Please help me out with a link to solutions :)

RGDS, Arie

Andie La-Rosa June 12, 2017 at 4:37 pmReply

Hi there Arie,

Not sure why you're having trouble seeing the confirmation email - you may want to do an "in:anywhere" search for the word "gravityscan" to see if you can find it, or speak to your mail provider to see what the issue might be.

We're working on extensive documentation about the HTTP Security Information results and how to decipher their meaning, but this is a pretty recent and thorough explanation with detailed information about what you can do to address these issues. One caveat: be careful about setting up the headers with broad "allow everything" rules - you may get an increase in severity warnings from the scans that way! For now, not having it set up at all is better than having it set up incorrectly. Hope that helps!

Petros May 20, 2017 at 11:01 amReply

Excellent work! Happy to have this service for joomla!
Just something weird. In three of my joomla sites i got a malware indication of /images/stories/3xp.php
However i ftped there and didnt find anything...
In addition when i added and you gravity php scanner and got a full scan it didnt find anything.

Keep up your great work!

zushiman May 18, 2017 at 3:32 pmReply

Two of my sites (one WP & one webshop) are blacklisted. The results are sure false positives. How can I remove these sites from blacklists??
By da way, appreciate your work for keeping us safe with Wordfence! Thanks!

mark May 18, 2017 at 4:50 pmReply

Unless it's justspam.org which we removed (they don't have a clear blacklist removal process) then it is likely that these are not false positives and your site is in fact blacklisted. Contact each list to find out how to remove your site.

David Bennett May 18, 2017 at 9:06 amReply

I am confused about a scan result that says I have a potentially malicious file detected in one of my sites at /wp-content/uploads/2016/05/

I FTP'd in and also looked at the tree within the file manager on the host directly - and both say that that directory is empty.

How can the scan detect a file in an empty directory?

mark May 18, 2017 at 4:52 pmReply

Hi David. This was a bug which is now fixed. Gravityscan is pointing out that you have public directory listing enabled which is not secure. But the language we used was erroneous - we described it as a malware file. That has now been fixed. Please try your scan again.

Dennis Spreen May 18, 2017 at 6:31 amReply

Why are you scrambling the agents code? This looks like an virus to me. I won't install that.

mark May 18, 2017 at 5:04 pmReply

Hi Dennis,

We base64 encode it so that you can transfer the file as plain text via FTP. During testing we had a few issues with some FTP clients transferring using text by default (ASCII transfer) and that broke the accelerator. So we solved that by base64 decoding. I'm starting a comment thread with the guys today to chat about open sourcing the accelerator because I think that will make it clear that it's something safe and you can also then view how we authenticate the Wordfence servers using public key encryption.

If anyone else would feel more comfortable if we open sourced Accelerator, please chime in here and I'll add your feedback to the conversation.

Mark.

Tyler James Bush May 18, 2017 at 4:40 amReply

I found that I had a plugin with a file in its folder with malware that I kept on installing to new sites. I have just removed the plugin for all my sites since it seems the plugin has not been updated and low rating. It was ultimate security checker for wordpress 2 star rating now.

mark May 18, 2017 at 5:05 pmReply

That's great news Tyler, thanks for the feedback.

MIComs May 18, 2017 at 2:38 amReply

Just the other day I was thinking, 'Hey how come there's no security plugin as awesome as Wordfence for Laravel?' And then the Wordfence guys make a security plugin that is awesome. Kudos, it makes me feel just that little bit safer.

But question, your scan works out of the box, but I don't suppose you'll be adding Laravel to the platforms you have specifics on? Given the scale at which the framework is growing on the web, you'll be covering a lot of bases.

mark May 18, 2017 at 2:45 amReply

You an use Gravity to do a really effective scan of Laravel if you install the agent.

Mark.

Tom ODea May 18, 2017 at 12:22 amReply

Thank you for providing this service. I ran a scan on one of my sites and found a Moderately Critical XSS Vulnerability in an old version of a WordPress plugin. We would not have known about this without GravityScan.
Thanks

mark May 18, 2017 at 2:46 amReply

That's so awesome Tom. Great to hear. Thanks for sharing. Will pass this on to the team.

Mark.

Andre Howard May 17, 2017 at 11:23 pmReply

Firstly, great job!!

Second, I've been working thru issues on one site I have that has been repeatedly hacked {don't know why as it a low volume site, but that a different issue}. I've been running various Wordpress plugins and external scans to help me get to the bottom of the outbreak and close the gap but have struggled to get there.

GravityScan did this on the first round - it found things that none of the other plugins did (not even WordFence), it's helped me dig deeper into my site and remove erroneous issues and vulnerabilities, now for once I feel like I'm on the up side.

So far, I'm impressed! this has been able to help me reduce impact to my sites and feel somewhat confident about what going on and enable me to win....

thanks again!!

mark May 18, 2017 at 2:46 amReply

Thanks Andre. Really great to hear. Will share this with the team. It means a lot when we have an impact like this.

Mark.

Doug May 17, 2017 at 11:17 pmReply

I recently started testing Wordfence on my sites, with the thought of upgrading to Wordfence Pro.

Gravityscan is a good move in general, HOWEVER, I am worried that you are purposely allowing confusion - in that you are avoiding the #1 and MOST OBVIOUS FAQ that will come from your existing Wordfence customers:

"With regard to my WordPress sites, Wordfence is giving me the same protection as Gravityscan correct?"

The answer to that questions should be upfront and center, BUT it is NOT.

Mark, I found a vague answer offered by you in a reply to a comment on a different Gravityscan blog post - and I am surprised.

Result: Using the best of my 'read-between-the-lines' ability, If I pay $10/mo. I will gain the function of getting my Apache install (at SiteGround) scanned for vulnerabilities?

The next obvious question is then, "What, if anything, is Wordfence scanning when set to scan outside my Wordpress environment?

Thank you,
Doug

mark May 18, 2017 at 2:48 amReply

Hi Doug,

Actually we anticipated this. I think we could have made it clearer where you can find the answer though. It's in our docs. I'm pretty sure I linked to it from our launch blog post, but I didn't link from today's blog post. This should help clear things up:

https://www.gravityscan.com/help/general-help-topics/gravityscan-complements-wordfence/

Mark.

Sergey May 17, 2017 at 11:15 pmReply

Mark.

Thank you for great online service. I use the Wordfence plugin on all my client websites and it helps to stay protected. I found out yesterday about the service Gravityscan of the email newsletter. I immediately signed up Gravityscan in order to see the service and evaluate its features. Team Gravityscan and Wordfence creates stunning and unique software products. Thank you for your work! New ideas, patience, success in overcoming challenges and energy!

mark May 18, 2017 at 2:48 amReply

Thanks Sergey, that's great feedback.

Mark.

John Rix May 17, 2017 at 10:35 pmReply

Nice service guys... gave me the kick up the pants I needed to get the Ubuntu PPA sorted out to upgrade my PHP version!

mark May 18, 2017 at 2:49 amReply

Awesome, thanks John!!

Cristian Balan May 17, 2017 at 10:11 pmReply

I run the latest Magento CE 1.9 (1.9.3.2) and the scan finds dozens of things patched a long time ago within previous versions.

E.g.
APPSEC-1484 Magento CE prior to 1.9.3 https://magento.com/security/patches/supee-8788
APPSEC-924 Magento CE prior to 1.9.2.0 https://magento.com/security/patches/supee-6285
APPSEC-921 Magento CE prior to 1.9.1.1 https://magento.com/security/patches/supee-5344-%E2%80%93-shoplift-bug-patch

Don't know what to do with it, to be honest as it seems a list of all Magento vulnerabilities that could apply to my site but not effectively exploitable since I have the latest version. Confused!?!

The fact that I had to whitelist the IP range in CloudFlare to be able to scan, is not really convincing.

I have also Gravityscan Accelerator active.
Any thoughts?

mark May 17, 2017 at 10:12 pmReply

Hi Cristian,

I think I approved your previous comment while you were drafting this one. Hit reload on the page and you should see both. Please see my previous reply.

Thanks.

Cristian Balan May 17, 2017 at 10:02 pmReply

I run the latest Magento CE 1.9 (1.9.3.2) and the scan finds dozens of things patched a long time ago within previous versions.

E.g.
APPSEC-1484 Magento CE prior to 1.9.3 https://magento.com/security/patches/supee-8788
APPSEC-924 Magento CE prior to 1.9.2.0 https://magento.com/security/patches/supee-6285
APPSEC-921 Magento CE prior to 1.9.1.1 https://magento.com/security/patches/supee-5344-%E2%80%93-shoplift-bug-patch

Don't know what to do with it to be honest.

Any idea?

mark May 17, 2017 at 10:03 pmReply

Thanks Christian,

Can you please msg the team on our contact form? If you share your scan with them that would help too I think.

Mark.

Hanson Bridgett May 17, 2017 at 9:48 pmReply

Looks like a great service. One thing that's confusing...I used my Analytics account to select sites to monitor, so they should already be verified, but I was told I'd need to verify them, and using the Google Analytics method fails. Shouldn't they be verified immediately, since they were selected directly from my Google Analytics list?

mark May 17, 2017 at 10:09 pmReply

Yes they should be. That sounds like a bug. Please report it on the contact page.

Thanks.

David Oster May 17, 2017 at 9:37 pmReply

Thank you people for this amazing product! The scans revealed a lot about how to fine tune both the servers and software!
Currently we are looking into updating our hosting pricing plans so we include the Pro version for every customer as a standard service with scheduled scans.

Keep up the good work,

David Oster aka George Pasparakis, CEO
https://eletter.gr

mark May 17, 2017 at 10:08 pmReply

Thanks David.

Mark May 17, 2017 at 9:27 pmReply

Am I getting these posts here - posted or seen? All that happens is the page scrolls up and nothing is seen. No thank-you message, no confirmation.

mark May 17, 2017 at 10:07 pmReply

Hi Mark,

All posts are moderated by me. So it just takes a little while for them to show up, usually with a reply from me.

Mark.

Brad Blosser May 17, 2017 at 9:26 pmReply

Hey Mark,
Question,

I've been running the free version of the wf firewall since before doing anything else on the website. I know the malware db doesn't update as often as the paid version but I can't afford the paid version.

The scan came back saying I have 3 vulnerabilities or malware of a highly critical nauture

the next one was in the wp-content/update folder
the next one was in the wp-includes/css folder
the next one was in the wp-includes/js folder

It doesn't say anywhere what the vulnerability or malware is nor where it is - it just points to the folder with a link?

Time and money are two commodities I don't have much of. Why is this new tool pointing out malware or other insecurities but not say what or where they are? I don't have money to pay for a malware removal if it's not necessary but I don't want to pass on malware to email recipients or viewers of my website if I do have it either.

Sigh... there's always something

mark May 17, 2017 at 10:07 pmReply

Hi Brad,

I think we have a fix for this that is being deployed today. It's caused by directory listing being enabled on parts of your site and Gravity sees a file listed in a directory and thinks it is a vulnerability. Should be fixed later today.

Mark.

Mark May 17, 2017 at 9:25 pmReply

I am not sure if this was part of your fixes, but on one of my scans, I got HTTP errors based on "404's, and "502's".

The listing of these errors does not help me resolve them because I have no idea, what or where on my site, these errors are generated?

I get the 502's, but the 404's (98 of them) leaves me blundered. If the scan tool would list these 404's, then I can begin dealing with them.

For 502's, it would be nice to know what the server was asked to do so maybe I can find out why the website server failed to respond.

Would that be possible? - or is that detail feature reserved for a premium account?

Thanks

mark May 17, 2017 at 10:04 pmReply

Hi Mark. No it's not reserved for Premium. I'll share your feedback with the team. Thanks.

Martin Thompson May 17, 2017 at 9:21 pmReply

Really impressed. Found a backdoor issue that was left by one of the devs working on a plugin. would have been easy for a hacker to find.

Am doing daily scans now!

Eveline Power May 17, 2017 at 9:13 pmReply

I added 6 sites yesterday and scanned them all. One shopping site is hosted by pinnaclecart.com but also has 3 wordpress sites in subfolders. I uploaded the Accelerator to the root but it isn't accessible so couldn't scan properly. I thought it might work if I put the php into the subfolders for the wordpress instances and register each as a separate site but it only registers the root domain, so I guess I'm out of luck in this case. The other site scans all worked really well - Thanks!

mark May 17, 2017 at 9:14 pmReply

Thanks Eveline. If you'd like you can use our contact form to file an issue with the team.

Glad to hear it worked on your sites.

Mark.

Lawrence May 17, 2017 at 8:54 pmReply

Hi
I did leave a comment but can't see it. Sorry if it ends up as a duplicate.
My comment was that I also run ithemes security and when I run the GravityScan I get a notification from ithemes that the GravityScan ip has been blocked. Don't know how this would affect the results but I would guess it would not be good.
Lawrence

mark May 17, 2017 at 9:01 pmReply

Hi Lawrence,

You can find out how to whitelist gravityscan here: https://www.gravityscan.com/help/scan-settings/whitelisting-gravityscan/

Mark.

Lawrence May 17, 2017 at 9:36 pmReply

Thanks Mark
Your response time to queries is remarkable.
Lawrence

mark May 17, 2017 at 10:07 pmReply

Thanks. Sometimes there's a delay. Like this one. :)

David Burton May 17, 2017 at 8:36 pmReply

When you notified me yesterday, I went right over and checked it out.. The results were very impressive and because of Gravityscan, I updated every part of my server and web site. Got the scanner running locally and it makes the scan far more detailed than running a scan from the web interface alone.

I am not a security guy, which is a good thing as I am just no good at it, but Gravityscan let me know what was out of date and what issues I was facing because of it.

Wordfence and Gravityscan did get into a fight over Gravityscan wanting to upload a file, but Wordfence won and this makes me happy :)

Keep up the awesome work guys.

mark May 17, 2017 at 9:02 pmReply

Thanks David!

John Palfrey May 17, 2017 at 8:31 pmReply

I found out with GravityScan that one of my websites was blacklisted and this was probably as a result of hacking more than 12 months ago. It accounts for the low traffic. I've been able to request it is removed, I wasn't even aware! Excellent -thanks very much.

G McClure May 17, 2017 at 8:59 pmReply

Same thing happened here. Discovered that one site I admin was on a blacklist due to shared hosting provider. Contacted provider with the details after getting some very quick help from Gravity Scan support on release day of all days!. Kudos to the entire team there at GravityScan.

mark May 17, 2017 at 9:00 pmReply

Thanks!! Glad to help.

mark May 17, 2017 at 8:32 pmReply

That's awesome news John. I'll share that with the team.

Mark.

Thomas May 17, 2017 at 8:27 pmReply

Super service, thanks.

mark May 17, 2017 at 8:27 pmReply

Thanks Thomas.

You must be a Gravityscan user to join the discussion. Register now or sign in if you already have an account.