Monday, June 18, 2018

Security Analytics At the Speed of Thought With ML and Elastic

Abstract: This talk was a continuation of the discussion started in February where we will overview how machine learning in Elastic X-Pack can be used to analyze data from a data lake help the SOC (Security Operations Center) and Threat Hunting teams find malicious actors in their environment. We will demonstrate how easy it is to pivot through data and start to expand the information we have around the compromise.

Geoff presented a demo similar to this one, https://www.elastic.co/blog/using-kibana-and-beats-for-security-analytics

May 23, 2018, was a great day! Thank you, all.

Wednesday, May 16, 2018

Searching Blockchain with FreeEed

The blockchain is composed of multiple blocks that can contain any information. However, it is not a database in a traditional sense: it is not fast and it does not answer queries.

For example, the speed of write is one block every 10 minutes for Bitcoin, and about one block every seven seconds for Ethereum. Queries, as such, do not exist at all: either SQL or NoSQL-type language is not provided.

Meanwhile, the information stored in Blockchain often needs to be searched. Here are the potential use cases

1. Any kind of eDiscovery with Blockchain needs a full search capability.
2. https://www.securedestate.io/ is an example of Ethereum-based for estate planning, together with legal and financial information.
3. There many examples of similar nature.

From today, such tool exists. FreeEed has been used by lawyers to do eDiscovery and legal review, and by researchers, for all kinds of investigations. It allows you to give it any kind of data as input (see here) and indexes that data for searches. The data can be loose Office files, PST mailboxes, a “load file” produced to lawyers as a result of an eDiscovery request, and as of now, Blockchain.

There is more: we are actively working on FreeEed all the time, adding input formats, processing capabilities, and machine learning. The tool is open source and welcomes new additions. The review part is called “FreeEed Review” and works through the browser.

The back end used to implement text search is Elasticsearch. This means that you can look at the processed data also through the powerful ELK (Elasticsearch, Logstash, Kibana) which are also open source.

Happy searching!

Monday, April 30, 2018

FreeEed with Elasticsearch (7.7.2 release)


Improvements in this version (7.7.2):
  • Elasticsearch integration. Now the users get more open source tools to work with FreeEed: Elasticsearch, Logstash, and Kibana. 
  • Bug fixes, code refactoring.
  • Go here http://freeeed.org/

Thursday, March 29, 2018

FreeEed 7.7.1 release

Here is what is new is FreeEed 7.7.1 release

  • Restored deduplication
  • Better email handling
  • Separated processing engine code into its own project
  • All UI forms done in IntelliJ, out of NetBeans and away from commercial editors
Enjoy!

Tuesday, September 12, 2017

Does FreeEed search for numbers? - Yes, it does!

This question was asked by one of the users, can he find numbers in the text that FreeEed indexes. I got curious myself and checked.

The reason that this is an important question is that I remember Craig Ball mentioning that in one of the requirements for good eDiscovery software. So OK, I ran a few searches and found out that out-of-the-box FreeEed does index all numbers. That felt good, and I am attaching the screenshots of the experiment.

Of course, that is not a special property of FreeEed but of Tika, Lucene, and SOLR. It's these components that are responsible for what FreeEed indexes.

Had this not been the case, I would tweak the use of the components, but luckily this was the way FreeEed already uses them. The advantage of passing through to these libraries is that the users can rely on the well-known Lucene syntax to do their searches.

Monday, July 31, 2017

Couchbase at Houston Hadoop & Spark Meetup

Justin Tuggle presented the well-justified reasons why today only NoSQL databases are up to par, to provide customer engagement and means for business survival, and of them, why Couchbase is to be preferred.

Here is the link to the materials.


Thursday, July 20, 2017

An easy way to run FreeEed on Amazon

Running FreeEed on Amazon is very easy and offers some substantial benefits.


  1. You can get a fully provisioned server in a minute
  2. You can get any size of hard drive and a large number of CPU
  3. It is as easy as using your desktop.
To start the server, find this AMI in the Oregon region on EC2: ami-e6acbf9f.

After you start the service, open the assigned IP in any browser. You will see a screen like the following below


















Click on the 'vnc.html'. You will see the login screen







After you log in, you will see a full Ubuntu desktop, where you can do any work. FreeEed is already installed.



Enjoy!