Using Bro to explore personal networks, like an AWS Wordpress blog

My personal Wordpress blog was hacked a few weeks ago. I hadn’t checked my simple AWS micro instance in a while, and fortunately, I happened to look at the site just a day after it was compromised. Wordpress makes it easy to stand up a pretty website if you don’t want to spend time crafting a front-end yourself, but security is a major question mark. If you keep it up to date and don’t load it with tons of third party content, it’s probably OK for a personal blog. However, I had several WP plugins and themes that were a few weeks behind the last update, as was WP Core. This is likely how the attacker was able to gain write-access to my one and only post on my blog and change the links to point at some sketchy-looking foreign domain websites to improve his SEO. Interestingly enough, the attacker also made some edits in the article, including grammatical fixes and clauses that were consistent with the content (environmental regulations).

Regardless, I updated everything, removed unnecessary plugins, added a plugin to record more detailed Wordpress history, secured my active accounts, and reverted the changes (except for one grammatical improvement - thank you attacker). Good Wordpress security and statistics plugins are difficult to come by, and they most likely increase the attack surface of the website, so I decided to opt for the tool I use and develop every day: Bro IDS. The visibility provided by Bro goes beyond threat detection and can be used as a more powerful version of netflow, showing what’s running on a system in significant detail.

Monitoring a single web server instance facing external http requests behind a pretty limiting firewall (inbound tcp/80 only, or tcp/22 from my static IP address only) is not our typical deployment for Bro here at Packetsled -- limited to basic web requests, and with no honeypot, I didn’t expect a whole lot of interesting attack vectors. However, there were some takeaways after letting Bro run for a few weeks, mostly regarding how my network operates and what kind of requests my site typically sees.

The blog uses basic unverified HTTP (I’m too lazy and cheap to set it up right), so I was unsure why I had at least one ssl log entry every hour:

#fields id.orig_h       id.orig_p       id.resp_h       id.resp_p       version cipher  curve   server_name     resumed      last_alert      next_protocol   established     cert_chain_fuids        client_cert_chain_fuids subject issuer  client_subject       client_issuer   validation_status

172.31.40.119   59718   52.201.183.155  443     TLSv12  TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384        secp256r1       -       F       -       -       F       Fb0mOL2796DAoCkRa3      (empty) -       -       -       -   -
172.31.40.119   41499   66.155.40.189   443     TLSv12  TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256        secp256r1       -       F       -       http/1.1        F       Fr4ISl3di8C0feHNC4,F2sfO14ZkvhwHhaNb4,F10O3d278YkNUhURTf     (empty) -       -       -       -       -

Ah of course -- after opening up the corresponding x509 certificate log, it’s clear that my device is reaching out to Wordpress about twice a day, looking for updates from “CN=*.wordpress.org,OU=Domain Control Validated CN=Go Daddy Secure Certificate Authority - G2,OU=http://certs.godaddy.com/repository/,O=GoDaddy.com.” More surprisingly though, my instance periodically was reaching out to an Amazon-based address every hour on my exposed network (as opposed to the back channels AWS uses to control instances):

pi@raspberrypi:~ $ whois 52.201.183.155

NetRange:       52.192.0.0 - 52.223.255.255
CIDR:           52.192.0.0/11
...
Organization:   Amazon Technologies Inc. (AT-88-Z)


x509.log
#fields certificate.version     certificate.serial                  certificate.subject     certificate.issuer      certificate.not_valid_before    certificate.not_valid_after     certificate.key_alg     certificate.sig_alg     certificate.key_type    certificate.key_length  certificate.exponent    certificate.curve       san.dns san.uri san.email       san.ip  basic_constraints.ca    basic_constraints.path_len

3       07      emailAddress=info@bitrock.com,CN=stats.bitnami.com,O=BitRock Update Services,L=Seville,ST=Seville,C=ES  emailAddress=info@bitrock.com,CN=BitRock Update Services,O=BitRock Update Services,L=Seville,ST=Seville,C=ES    1284478570.000000       1599838570.000000       rsaEncryption   sha1WithRSAEncryption   rsa     1024    65537   -       -       -       -       -       F       -
3       280392F6950C4C  CN=*.wordpress.org,OU=Domain Control Validated  CN=Go Daddy Secure Certificate Authority - G2,OU=http://certs.godaddy.com/repository/,O=GoDaddy.com\\, Inc.,L=Scottsdale,ST=Arizona,C=US        1417824798.000000       1513368681.000000       rsaEncryption   sha256WithRSAEncryption rsa     2048    65537   -       *.wordpress.org,wordpress.org   -       -       -       F       -

Again using Bro’s x509 log to extract more information about the certificate owner, this is Bitnami, whose VM and tech stack I used to set up my Wordpress instance.



It resolves to stats.bitnami.org. That must have been something I missed in the ‘Terms and Conditions’.

The dhcp.log was about what you would expect with a rarely changing IP, but some of the DNS queries my machine made were interesting. Most of them are pretty clear: ‘security.ubuntu.com’ for updates, ‘stats.bitnami.org’ for my newly discovered stats reporting mechanism, and plenty of internal AWS domain server lookups. Others were less clear, such as a large number of queries for my own website name www.leolinsky.com (never figured out that one, but I assume it’s related to one of my Wordpress plugins), and a Google lookup ( crawl-66-249-79-106.googlebot.com -- also related to one of my statistics and monitoring plugins, I believe)

dns.log
#path   dns
#fields id.orig_h       id.orig_p       id.resp_h       id.resp_p       proto   trans_id        rtt     query   qclass       qclass_name     qtype   qtype_name      rcode   rcode_name      AA      TC      RD      RA      Z       answers TTLs    rejected

172.31.40.119   59480   172.31.0.2      53      udp     2863    -       254.118.225.73.in-addr.arpa  -       -       -       -       0       NOERROR F       F       F       T       0       c-73-225-118-254.hsd1.wa.comcast.net 60.000000       F
172.31.40.119   52552   172.31.0.2      53      udp     42576   -       2.0.31.172.in-addr.arpa      -       -       -       -       0       NOERROR F       F       F       T       0       ip-172-31-0-2.us-west-2.compute.internal     60.000000       F
172.31.40.119   60949   172.31.0.2      53      udp     48886   -       114.79.249.66.in-addr.arpa   -       -       -       -       0       NOERROR F       F       F       T       0       crawl-66-249-79-114.googlebot.com    60.000000       F
172.31.40.119   36634   172.31.0.2      53      udp     35400   -       www.leolinsky.com    -       -       -       -       0       NOERROR F       F       F       T       0       35.162.195.120  60.000000       F
172.31.40.119   49864   172.31.0.2      53      udp     30057   -       stats.bitnami.org    -       -       -       -       0       NOERROR F       F       F       T       0       52.201.183.155  60.000000       F
172.31.40.119   44066   172.31.0.2      53      udp     39877   -       security.ubuntu.com  -       -       -       -       0       NOERROR F       F       F       T       0

The http log is where it quickly became clear that no one visits my website except bots, malicious entities, and myself. Note that my instance’s internal IP is 172.31.40.119.

http.log 
#fields id.orig_h       id.orig_p       id.resp_h       id.resp_p       trans_depth     method  host    uri     referrer     version user_agent      request_body_len        response_body_len       status_code     status_msg      info_code       info_msg     tags    username        password        proxied orig_fuids      orig_filenames  orig_mime_types resp_fuids      resp_filenames       resp_mime_types

35.162.195.120  39028   172.31.40.119   80      1       POST    leolinsky.com   /wp-cron.php?doing_wp_cron=1488229413.0440499782562255859375 http://leolinsky.com/wp-cron.php?doing_wp_cron=1488229413.0440499782562255859375     -       WordPress/4.7.2; http://leolinsky.com   0       0       -       -       -       -       (empty) -       -       -   --       -       -       -       -
172.31.40.119   49929   169.254.169.254 80      1       -       -       -       -   1.0      -       0       8       200     OK      -       -       (empty) -       -       -       -       -       -       FIOPkb38ZkF8rKEOel   -       -
172.31.40.119   49930   169.254.169.254 80      1       -       -       -       -   1.0      -       0       345     404     Not Found       -       -       (empty) -       -       -       -       -       -       FZXQ67dF6Wx9MbUrh    -       text/html
35.162.195.120  39047   172.31.40.119   80      1       POST    www.leolinsky.com   /wp-cron.php?doing_wp_cron=1488230712.3863990306854248046875     http://www.leolinsky.com/wp-cron.php?doing_wp_cron=1488230712.3863990306854248046875 -       WordPress/4.7.2; http://www.leolinsky.com       0       0       -       -       -       -       (empty) -   --       -       -       -       -       -       -
66.249.79.110   56031   172.31.40.119   80      1       GET     www.leolinsky.com   /2016/11/        -       -       Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)        0       0       -   --       -       (empty) -       -       -       -       -       -       -       -       -
180.76.15.27    50507   172.31.40.119   80      1       GET     www.leolinsky.com   /tag/non-aggression/     -       -       Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)     0   0-       -       -       -       (empty) -       -        -       -       -       -       -       -       -
92.85.68.75     57131   172.31.40.119   80      4       POST    www.leolinsky.com   /wp-login.php    -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        51      0       -   --       -       (empty) -       -       -       FgaxLN10GdbeKnj1db      -       text/plain      -       -       -
81.11.190.250   65495   172.31.40.119   80      2       POST    www.leolinsky.com   /xmlrpc.php      -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        216     0       -   --       -       (empty) -       -       -       FvY2Ce1KqOaUXXiiTf      -       application/xml -       -       -
81.11.190.250   65495   172.31.40.119   80      3       GET     www.leolinsky.com   /wp-login.php    -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        0       0       -   --       -       (empty) -       -       -       -       -       -       -       -       -
81.11.190.250   65495   172.31.40.119   80      5       GET     www.leolinsky.com   /wp-login.php    -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        0       0       -   --       -       (empty) -       -       -       -       -       -       -       -       -
81.11.190.250   65495   172.31.40.119   80      1       POST    www.leolinsky.com   /xmlrpc.php      -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        216     0       -   --       -       (empty) -       -       -       F3EeSQyDVfg8BS47k       -       application/xml -       -       -
81.11.190.250   65495   172.31.40.119   80      4       POST    www.leolinsky.com   /wp-login.php    -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        50      0       -   --       -       (empty) -       -       -       Ftq19Q1J1z3ik6f3N4      -       text/plain      -       -       --       -       -
172.31.40.119   38934   35.162.195.120  80      1       -       -       -       -   1.1      -       0       0       200     OK      -       -       (empty) -       -       -       -       -       -       -       -   -
89.211.211.103  49616   172.31.40.119   80      2       POST    www.leolinsky.com   /xmlrpc.php      -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        217     0       -   --       -       (empty) -       -       -       FKL6zw4tA23qNtOcK5      -       application/xml -       -       -
195.230.145.22  47823   172.31.40.119   80      1       GET                 35.162.195.120  /   --       curl/7.17.1 (mips-unknown-linux-gnu) libcurl/7.17.1 OpenSSL/0.9.8i zlib/1.2.3   0       0       -       -       -       -   (empty)  -       -       -       -       -       -       -       -       -

We can see how Chrome identifies itself from my system via the user-agent string while I was logged into my website ( this can be linked to a Chrome version; the OS information is irrelevant ).

x.x.x.x  48144   172.31.40.119   80      1       GET     www.leolinsky.com       /wp-admin/admin-ajax.php?action=SimpleHistoryNewRowsNotifier&apiArgs[since_id]=146&apiArgs[dates]=lastdays:30&ip-geo-block-auth-nonce=f8bc15686e        http://www.leolinsky.com/wp-admin/      -      Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36    0       0       -       -       -       -       (empty)--       -       -       -       -       -       -       -

Here’s an example of an IP that attempted to login to my site, according to a WP history plugin:

99.20.91.64     52001   172.31.40.119   80      1       POST    www.leolinsky.com       /xmlrpc.php     -       -       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1        215     0       -       -       -       -      (empty)  -       -       -       Fsqr162lYxheinQ95g      -       application/xml -       -       -



This attacker also triggered some “Weird” logs with malformed TCP flows, perhaps attempting known out-dated WP vulnerabilities to gain access:

99.20.91.64     52001   172.31.40.119   80      bad_TCP_checksum
99.20.91.64     52001   172.31.40.119   80      data_before_established
99.20.91.64     52001   172.31.40.119   80      inappropriate_FIN
99.20.91.64     52049   172.31.40.119   80      data_before_established
99.20.91.64     52049   172.31.40.119   80      inappropriate_FIN

I noticed a large number of traffic from the unroutable address 169.254.169.254, which was interesting because it included HTTP requests, even triggering 404’s. I’m not sure what Amazon was doing there, although it used a second Amazon-owned IP as well (35.162.195.120). Amazon is known to alert users when Bitcoin mining is detected on their instances (a common use case of a compromised account), so it’s certainly possible that they do other web monitoring as well.

There were also a number of foreign-originated tcp sessions, most of which seemed like bots (the patterns become obvious when plotted):



In summary, there’s Bro’s conn log with its exhaustive list of netflow-like entries. I was interested in the longest flow (sorted with awk 'NR > 4' < conn.log | sort -t$'\t' -k 9 -n from bro.org) which was a 30 second long AWS tcp flow which generated all kinds of weird logs.

x.x.x.x 44675   172.31.40.119   80      tcp     http    4.120405        1186    0       SH      F       T      0ScDAF   6       1438    0       0       (empty)
x.x.x.x  41411   172.31.40.119   80      tcp     http    4.937924        1186    0       SH      F       T      0ScDAF   6       1438    0       0       (empty)
172.31.40.119   60090   91.189.88.162   80      tcp     -       7.811027        0       984617  SHR     T       F      0^hCacdf 0       0       43      946737  (empty)
172.31.40.119   39107   54.71.119.81    80      tcp     -       30.188989       0       2746583 SHR     T       F      0^hCdcaf 0       0       884     2535428 (empty)

bitnami@ip-172-31-40-119:/usr/local/bro/logs/current$
bitnami@ip-172-31-40-119:/usr/local/bro/logs/current$ grep -rnw "54.71.119.81"
files.log:18: 54.71.119.81    172.31.40.119   Cw2Zz81Z7gWRzH8Lph      HTTP    0       MD5,SHA1        text/html       -      0.000000 F       F       325     325     0       0       F       -       3db7e12da4a623a3a10784b27db3d48f        b785d991b655989751ce05841301484d0c636f1a        -      -
files.log: 54.71.119.81    172.31.40.119   Cw2Zz81Z7gWRzH8Lph      HTTP    0       MD5,SHA1        text/plain      -      0.000000 F       F       31562   65876   0       0       F       -       0b132e685181d75acc5ec0623996868c        04f782fff2f7a5e194cba0b147399923e44dbc54        -      -
conn.log: 172.31.40.119   39107   54.71.119.81    80      tcp     -       -       -       -       OTH     T       F      0C       0       0       0       0       (empty)
conn.log: 172.31.40.119   39107   54.71.119.81    80      tcp     -       30.188989       0       2746583 SHR     T      F0       ^hCdcaf 0       0       884     2535428 (empty)
http.log: 172.31.40.119   39107   54.71.119.81    80      1       -       -       -       -       1.1     -       0      325      404     Not Found       -       -       (empty) -       -       -       -       -       -       FufUJjC7AjvIX9Xh3       -       text/html
http.log: 172.31.40.119   39107   54.71.119.81    80      2       -       -       -       -       1.1     -       0      31562    200     OK      -       -       (empty) -       -       -       -       -       -       FGd2Jb1Y3F3Pa6y9va      -       text/plain
weird.log: 172.31.40.119   39107   54.71.119.81    80      active_connection_reuse -       F       bro
weird.log: 172.31.40.119   39107   54.71.119.81    80      truncated_tcp_payload   -       F       bro
weird.log: 172.31.40.119   39107   54.71.119.81    80      above_hole_data_without_any_acks        -       F       bro

And connection summary put out by Bro:

>== Total === 2017-02-27-00-00-03 - 2017-02-27-00-58-20

   - Connections  51.0 - Payload 12.6k -
 Ports        | Sources                   | Destinations              | Services           | Protocols | States        |
 80     62.7% | 172.31.40.119#1     80.4% | 169.254.169.254#2   31.4% | -            49.0% | 6   66.7% | SHR     56.9% |
 53     27.5% | 35.162.195.120#3     5.9% | 172.31.0.2#4        27.5% | dns          27.5% | 17  33.3% | OTH     23.5% |
 67      5.9% | 79.31.239.12#5       3.9% | 172.31.40.119#6     19.6% | http         17.6% |           | SH      17.6% |
 443     3.9% | 66.249.64.110#7      3.9% | 35.162.195.120#8    11.8% | dhcp          5.9% |           | S0       2.0% |
              | 68.180.230.223#9     2.0% | 172.31.32.1#10       5.9% |                    |           |               |
              | 66.249.64.163#11     2.0% | 52.201.183.155#12    3.9% |                    |           |               |
              | 66.249.64.114#13     2.0% |                           |                    |           |               |


#1=ip-172-31-40-119.us-west-2.compute.internal  
#2=instance-data.us-west-2.compute.internal  #3=ec2-35-162-195-120.us-west-2.compute.amazonaws.com
#4=ip-172-31-0-2.us-west-2.compute.internal  #5=host12-239-dynamic.31-79-r.retail.telecomitalia.it  #6=ip-172-31-40-119.us-west-2.compute.internal
#7=crawl-66-249-64-110.googlebot.com  #8=ec2-35-162-195-120.us-west-2.compute.amazonaws.com
#9=b115511.yse.yahoo.net
#10=ip-172-31-32-1.us-west-2.compute.internal
#11=crawl-66-249-64-163.googlebot.com  #12=ec2-52-201-183-155.compute-1.amazonaws.com
#13=crawl-66-249-64-114.googlebot.com

Bro is a powerful tool. However, the raw data can still be overwhelming -- many of these log excerpts were already heavily redacted for readability. At Packetsled, we build and rank our detections to filter out the noise and further decorate Bro’s statistics, allowing both visibility and real-time threat detection.

PacketSled

Leo Linsky
Software Engineer
PacketSled


Download WPS Office for Windows today!

Download

Download WPS Office for iOS and Android today!

Find us on the iTunes AppStore
Get it on Google Play

Guest Blogger

When friends of WPS Office add their voices to our blog, you can find their information at the bottom of the post they have written!