Felix Schupp: New kid on the block: The BlackRay Data Engine
authorDobrica Pavlinusic <dpavlin@rot13.org>
Mon, 30 Nov 2009 20:28:25 +0000 (21:28 +0100)
committerDobrica Pavlinusic <dpavlin@rot13.org>
Mon, 30 Nov 2009 20:51:25 +0000 (21:51 +0100)
media/froscon2009-BlackRey_Data_Engine/hCalendar.html [new file with mode: 0644]
media/froscon2009-BlackRey_Data_Engine/presentation.pdf [new symlink]
media/froscon2009-BlackRey_Data_Engine/video.ogv [new symlink]
media/froscon2009-BlackRey_Data_Engine/video.srt.yaml [new file with mode: 0644]
media/froscon2009-BlackRey_Data_Engine/www.sh [new file with mode: 0755]

diff --git a/media/froscon2009-BlackRey_Data_Engine/hCalendar.html b/media/froscon2009-BlackRey_Data_Engine/hCalendar.html
new file mode 100644 (file)
index 0000000..6fdd6cf
--- /dev/null
@@ -0,0 +1,18 @@
+<!-- http://microformats.org/wiki/hcalendar-cheatsheet -->
+<div class="vevent">
+ <span class="organiser">
+Felix Schupp
+ </span>
+ <span class="summary">
+New kid on the block: The BlackRay Data Engine
+ </span>
+ <a class="url" href="http://programm.froscon.org/2009/events/462.en.html">FrOSCon</a>,
+ <abbr class="dtstart" title="20090822">2009-08-22</abbr>
+ <div class="description">
+The BlackRay Data Engine is a high-performance, in-memory relational database, designed for large data sets and high performance. Originally designed for directory applications it offers features such as token search, token position, phonetic search and the combination of these features with leading, trailing and mid-span wildcards. It is built to run on standard hardware, but offer the ability to index data in the over 100 Million row range, with constant search throughput of several 100 queries per second, even with complex queries. This talk will cover the basic index internal design, the overall data engine setup, and options such as the available query- and management APIs.
+<p>
+In this talk we would like to explain our motivations for designing and building BlackRay, and then elaborate on the architecture of the internals of the data engine. The index structures inside the data engine are designed for low memory consumption, and the ability to quickly index large amounts of data. A total of five index layers, also called index perspectives, is required to fulfill the functions required for our search algorithms. Our smart combination of binary- and permuterm based search offers significant performance benefits over many traditional tree- and trie-based searches. Finally, searching for a token combination within a single table column only result in linear complexity, rather than exponential as in most typical index structures. The option to additionally compress the index further reduce the amount of memory used during operation.
+<p>
+Additionally we will explain the overall data engine setup, including the object-oriented search API (Java, C++, Pyhton, C#), as well as the progress on the PostgreSQL compatible socket interface for JDBC and ODBC. Further topics covered in our brief introduction are Remora, the web-based management interface, and the fail-over and recovery mechanisms of BlackRay, including the options available to maintain a history of your data.
+ </div>
+</div>
diff --git a/media/froscon2009-BlackRey_Data_Engine/presentation.pdf b/media/froscon2009-BlackRey_Data_Engine/presentation.pdf
new file mode 120000 (symlink)
index 0000000..a8a9138
--- /dev/null
@@ -0,0 +1 @@
+102_blackray.pdf
\ No newline at end of file
diff --git a/media/froscon2009-BlackRey_Data_Engine/video.ogv b/media/froscon2009-BlackRey_Data_Engine/video.ogv
new file mode 120000 (symlink)
index 0000000..55a6945
--- /dev/null
@@ -0,0 +1 @@
+youtube_Z8xGm6cQhWc.flv
\ No newline at end of file
diff --git a/media/froscon2009-BlackRey_Data_Engine/video.srt.yaml b/media/froscon2009-BlackRey_Data_Engine/video.srt.yaml
new file mode 100644 (file)
index 0000000..57e4e1a
--- /dev/null
@@ -0,0 +1,172 @@
+---
+- 0.9
+- 1.9
+- '[1] 1'
+---
+- 27.3
+- 28.3
+- '[2] Presentation Agenda'
+---
+- 78.2
+- 79.2
+- '[3] Brief BlackRay History'
+---
+- 80.9
+- 81.9
+- '[4] What is BlackRay?'
+---
+- 135.6
+- 136.6
+- '[5] BlackRay History'
+---
+- 209.5
+- 210.5
+- '[6] Why Build Another Database?'
+---
+- 324.4
+- 325.4
+- '[7] Options available 1999'
+---
+- 400.5
+- 401.5
+- '[8] Options evaluated in 2005'
+---
+- 524.7
+- 525.7
+- '[9] Decision to Implement BlackRay'
+---
+- 583.9
+- 584.9
+- '[10] Current Release'
+---
+- 657.9
+- 658.9
+- '[11] Technology Overview'
+---
+- 661.4
+- 662.4
+- '[12] Why call it Data Engine?'
+---
+- 710.0
+- 711
+- '[13] Why call it Data Engine?'
+---
+- 783.8
+- 784.8
+- '[14] BlackRay Architecture'
+---
+- 985.1
+- 986.1
+- '[15] Hierarchical Model'
+---
+- 1061.8
+- 1062.8
+- '[16] Getting Data Into BlackRay'
+---
+- 1184.2
+- 1185.2
+- '[17] Basic Load Performance Data'
+---
+- 1554.8
+- 1555.8
+- '[18] Data Universe'
+---
+- 1727.4
+- 1728.4
+- '[19] Snapshots and Data Versioning'
+---
+- 1822.1
+- 1823.1
+- '[20] Transactions in BlackRay'
+---
+- 1920.7
+- 1921.7
+- '[21] Query APIs'
+---
+- 1998.9
+- 1999.9
+- '[22] Query APIs'
+---
+- 2072.5
+- 2073.5
+- '[23] Management Features'
+---
+- 2122.1
+- 2123.1
+- '[24] Administration Requirements'
+---
+- 2225.1
+- 2226.1
+- '[25] Some more details'
+---
+- 2345.7
+- 2346.7
+- '[26] Positioning BlackRay'
+---
+- 2347.7
+- 2348.7
+- '[27] BlackRay and other Projects'
+---
+- 2365.1
+- 2366.1
+- '[28] OS Projects with similar Goals'
+---
+- 2559.1
+- 2560.1
+- '[29] Commercial Alternatives'
+---
+- 2634.8
+- 2635.8
+- '[30] Is it the right thing for me?'
+---
+- 2704.9
+- 2705.9
+- '[31] BlackRay will fit you well when...'
+---
+- 2766.8
+- 2767.8
+- '[32] Project Roadmap'
+---
+- 2769.0
+- 2770
+- '[33] Immediate Roadmap'
+---
+- 2817.1
+- 2818.1
+- '[34] Midterm Roadmap'
+---
+- 2879.2
+- 2880.2
+- '[35] Longterm Roadmap'
+---
+- 2967.6
+- 2968.6
+- '[36] The Team behind BlackRay'
+---
+- 2968.9
+- 2969.9
+- '[37] Development Team'
+---
+- 2997.3
+- 2998.3
+- '[38] Thomas Wunschel'
+---
+- 3020.9
+- 3021.9
+- '[39] Mike Alexeev'
+---
+- 3053.5
+- 3054.5
+- '[40] Felix Schupp'
+---
+- 3091.0
+- 3092
+- '[41] Wrap-Up'
+---
+- 3094.9
+- 3095.9
+- '[42] What to do next'
+---
+- 3180.6
+- 3181.6
+- '[43] Contact Us'
diff --git a/media/froscon2009-BlackRey_Data_Engine/www.sh b/media/froscon2009-BlackRey_Data_Engine/www.sh
new file mode 100755 (executable)
index 0000000..ab910e8
--- /dev/null
@@ -0,0 +1,2 @@
+wget http://programm.froscon.org/2009/attachments/102_blackray.pdf
+cclive http://www.youtube.com/watch?v=Z8xGm6cQhWc