Lobster
The LOBSTER project emerged from the SCAMPI initiative active during 2004 and 2005. This earlier European effort aimed to create a scalable monitoring platform for the Internet. LOBSTER itself received funding from the European Commission and concluded operations in 2007. Its primary goal involved building an advanced pilot infrastructure based on passive network monitoring sensors. The team sought to develop novel performance and security applications enabled by this new data availability. They also created specific tools to anonymize traffic data and prevent unauthorized access or tampering.
LOBSTER relied on passive network traffic monitoring rather than active probing methods. Instead of collecting flow-level summaries, the system recorded all IP packets flowing through monitored links. These records included both headers and payloads for complete information capture. This approach allowed researchers to tackle monitoring problems with greater accuracy than flow-based statistics. Active monitoring techniques often miss details that passive recording captures naturally. The method provided a full picture of actual network traffic without interfering with normal operations.
Developers built monitoring applications using the MAPI framework known as the Monitoring Application Programming Interface. M Polychronakis and colleagues published design details for this interface in 2004 within the context of SCAMPI and LOBSTER projects. Programmers could express complex monitoring needs while choosing only the information they required. This balance between overhead and data retrieval became a core feature of the architecture. The interface supported remote distributed applications receiving data from multiple sensors simultaneously. It enabled flexible customization based on the specific requirements of each participating organization.
Thirty-six sensors operated across nine countries throughout Europe by various organizations. At any given moment the system monitored traffic spanning over 2.3 million IP addresses. Multiple groups deployed these sensors to gather diverse network data points. The infrastructure fed into IST 2.3.5 Research Networking testbeds aimed at improving European internet infrastructure. A press release dated the 17th of May 2007 documented the scale of this deployment effort. The widespread geographic distribution allowed comprehensive coverage of international traffic flows.
The system successfully identified more than 400,000 Internet attacks during its operational lifespan. These detections occurred through automated analysis of captured packet data without human intervention. Security researchers utilized the complete packet records to identify malicious patterns and threats. The high volume of detected incidents demonstrated the effectiveness of passive monitoring for threat detection. Organizations relied on these findings to improve their own network defenses significantly. The project proved that large-scale passive observation could yield actionable security intelligence.
Funding came directly from the European Commission until the project concluded in 2007. LOBSTER ceased operations after achieving all stated goals regarding infrastructure and applications. The initiative contributed valuable insights to the broader field of network research. Several organizations continued using developed tools like Appmon and Stager after official termination. ABW remained another application built upon the DiMAPI library for distributed tracking. The legacy of the project persisted through these specialized measurement systems used by participating institutions.
Up Next
Common questions
What was the LOBSTER project and when did it operate?
The LOBSTER project emerged from the SCAMPI initiative active during 2004 and 2005 and concluded operations in 2007. It received funding from the European Commission to build an advanced pilot infrastructure based on passive network monitoring sensors.
How does the LOBSTER system collect data compared to other methods?
LOBSTER relies on passive network traffic monitoring rather than active probing methods to record all IP packets flowing through monitored links. This approach captures both headers and payloads for complete information capture without interfering with normal operations.
Who developed the MAPI framework used by the LOBSTER team?
M Polychronakis and colleagues published design details for the Monitoring Application Programming Interface known as MAPI in 2004 within the context of SCAMPI and LOBSTER projects. Programmers could express complex monitoring needs while choosing only the information they required using this interface.
Where were the thirty-six LOBSTER sensors deployed across Europe?
Thirty-six sensors operated across nine countries throughout Europe by various organizations at any given moment. The system monitored traffic spanning over 2.3 million IP addresses and fed into IST 2.3.5 Research Networking testbeds aimed at improving European internet infrastructure.
What security results did the LOBSTER project achieve during its lifespan?
The system successfully identified more than 400,000 Internet attacks during its operational lifespan through automated analysis of captured packet data. Security researchers utilized the complete packet records to identify malicious patterns and threats without human intervention.