Mundaú - Distributed Computing (or not)

This blog is moving

2011-10-26T15:41:00.000-07:00

Dear reader,

This blog is moving to a new address. Please update your subscriptions to: http://blogs.ubc.ca/elizeu

3MT Competition

2011-03-23T10:32:00.001-07:00

Recently, the ECE Department joined a Campus-wide initiative: the Three Minutes Thesis Competition.

It was a really fun event (see 3MT Competition Winner)! More importantly, we had the chance to discuss the projects other graduate students are working on. I'm looking forward for the next round.

Theory Tuesdays -- What do Wikipedia, car-pooling, couch surfing, and social bookmarking have in common?

2010-12-01T01:54:00.001-08:00

Last fall, I worked for four months in Google, Zürich. It was a fun and enriching experience, indeed. The opportunity to learn and contribute to the technology that is used by millions of people daily is quite exciting.

Besides enjoying the Google-life, during my days in Zürich, I was invited by two artist friends, Silvan Käelin [1,2,3] and Philip Matesic [4,5,6]), to give a talk about my PhD thesis research at Perla Mode -- as part of the Theory Tuesdays project.

Philip organizes a weekly event at Perla Mode named Theory Tuesdays. The goal is to bring together artists, researchers (of multiple disciplines), and the public to discuss a variety of topics such as art, technology, society, and their intersection.

I received the invitation with surprise and interest. It was a new experience to talk to an audience completely outside my field of research. Also, it was a chance to receive feedback (from such non-technical group) about what I think it is a relevant topic of study.

The idea was to have a middle ground between what I have been investigating (i.e., techniques to assess the value of contributions in peer production systems [pdf]) and a the broader topic that could interest the attendance.

Therefore, it seemed appropriate to introduce the notion of peer production systems [7], hint the questions I am interested in answering in this context, and asking the participants related questions such as: how do they perceive the value of information they consume online? do they often perceive themselves contributing to others by producing information online? what is the main incentive to do so? What are the aspects they take into account to decided whether an information provider produces value to them?

I am glad that the "talk" turned into a lively conversation about about all these questions and other aspects related to online peer production. We covered topics from the basic notion of social production (and why it works so well in certain scenarios), passed through specifics about the utility of tagging (e.g., classification languages may emerge through collaboration), and talked about the intuition behind the techniques I am designing to assess the value of contributions in social tagging systems.

Although anecdotal, it was possible to observe from the discussion two explicit trends on the perception of value of online peer-produced information: novelty and trust on the information producer. These aspects came up in the discussions as crucial to the users to assess the value of peer-produced information. It is important to note that the information consumer's interest is an implicitly aspect considered in the value assessment.

The observations are somehow intuitive, but for me it was quite important and helpful to have a first-hand discussion with the real users of the systems I study. It does help one to tune the questions to ask and where the relevancy of one's research. I hope to have more opportunities like this. Thanks to Silvan and Philip for the first one.

References:

[1] One Man
[2] Lagoa do Ouro
[3] Temps de Poussiére (Time of Dust)
[4] An Bonus
[5] Mau Series
[6] To Don Pedro with Mr. Gonzalez
[7] Y. Benkler. "The Wealth of Networks: How Social Production Transforms Markets and Freedom"

The Small World of File Sharing

2010-10-01T02:34:00.000-07:00

The IEEE TPDS has recently published a preprint of an interesting piece of work that I had the pleasure to collaborate with.

Adriana Iamnitchi, Matei Ripeanu, Elizeu Santos-Neto, Ian Foster, "The Small World of File Sharing," IEEE Transactions on Parallel and Distributed Systems, 28 Sept. 2010. IEEE computer Society Digital Library. IEEE Computer Society.

Abstract:

Web caches, content distribution networks, peer-to-peer file sharing networks, distributed file systems, and data grids all have in common that they involve a community of users who use shared data. In each case, overall system performance can be improved significantly by first identifying and then exploiting the structure of community's data access patterns. We propose a novel perspective for analyzing data access workloads that considers the implicit relationships that form among users based on the data they access. We propose a new structure —the interest-sharing graph— that captures common user interests in data and justify its utility with studies on four data-sharing systems: a high-energy physics collaboration, the Web, the Kazaa peer-to-peer network, and a BitTorrent file-sharing community. We find small-world patterns in the interest-sharing graphs of all four communities. We investigate analytically and experimentally some of the potential causes that lead to this pattern and conclude that user preferences play a major role. The significance of small-world patterns is twofold: it provides a rigorous support to intuition and it suggests the potential to exploit these naturally emerging patterns. As a proof of concept, we design and evaluate an information dissemination system that exploits the small-world interest-sharing graphs by building an interest-aware network overlay. We show that this approach leads to improved information dissemination performance.

Assessing the Value of Contributions in Tagging Systems

2010-07-08T06:27:00.000-07:00

During the past two and a half months, I have been visiting the InWeb/UFMG at Belo Horizonte - MG - Brazil. Besides enjoying the great 'Cozinha Mineira', in this opportunity, we studied the problem of assessing the value of contributions in social tagging systems, and took the first steps towards a solution. The ideas will be presented in an article at the 2nd IEEE International Symposium on Social Intelligence and Networking (SIN-10) in 20-22August 2010. Next stop: to give a talk at the Laboratório de Sistemas Distribuídos (UFCG).

Abstract -- Assessing the value of individual users' contributions in peer-production systems is paramount to the design of mechanisms that support collaboration and improve users’ experience. For instance, to incentivize contributions, file-sharing systems based on the BitTorrent protocol equate value with volume of contributed content and use a prioritization mechanism to reward users who contribute more. This approach and similar techniques used in resource-sharing systems rely on the fact that the physical resources shared among users are easily quantifiable. In contrast, information-sharing systems, like social tagging systems, lack the notion of a physical resource unit (e.g., content size, bandwidth) that facilitates the task of evaluating user contributions. For this reason, the issue of estimating the value of user contributions in information sharing systems remains largely unexplored. This paper introduces this problem and takes the first steps towards a solution. More precisely, it presents a framework to design algorithms that estimate the value of user contributions in tagging systems, proposes three complementary success criteria for potential solutions, and outlines the methodological evaluation challenges.

Object-to-Object Similarity

2010-01-29T15:39:00.000-08:00

Recently, I read a paper published at VLDB'2008 titled: Accuracy Estimates and Optimization Techniques for SimRank Computation by Lizorkin et al. [1].

In summary, the paper studies a specific algorithm -- SimRank -- that determines similarity scores between objects by augmenting an available link structure that connects the objects (e.g., hyperlinks between pages, friendship links in online social networks, similar tags between photos in Flickr). The intuition behind SimRank is that "two objects are similar if they are referred to by similar objects".

As the title indicates, the work contributions can be divided into two parts: 1) it provides accuracy estimates for the iterative computation of SimRank scores; and, 2) it proposes and analysis three optimization techniques that reduces the computational complexity of the original algorithm from O(n^4) to O(n^3).

One point, which seems to need a deeper investigation, is whether the similarity scores produced by SimRank have high quality (by quality I mean they do capture the true similarity between objects). Indeed, the original SimRank work by Jeh & Widom [2] and Lizorkin's paper lack a discussion (and experiments) on the quality of scores.

Nevertheless, the beauty of Lizorkin's paper resides on the fact that the proposed optimization techniques may be helpful in other contexts, though they originate from a specific context -- improving SimRank. In the following, I briefly describe the intuition behind each optimization.

Essential nodes -- by exploiting the graph structure, the authors are able to define the set of essential nodes of a particular node v, and more importantly, to prove that if a given node b is outside the set of essential nodes of v, the similarity score between them is zero. It turns out, that computing the set of essential nodes is much cheaper than computing the similarity score. Hence, it leads to the first optimization.

Partial sums -- SimRank similarity is recursive by definition. This implies that the similarity score of between some pairs may be reused (without recomputation) to compute the similarity between several other pairs of nodes. The idea applied here was to use memoization to reduce the cost of computing parts of the sum in the computation of a final score. It occurs to me that memoization may be good even in a dynamic graph (i.e., a graph that changes over time), depending, of course, on the trade off between accuracy of the scores and the computational cost implied by the recomputation.

Threshold-sieved similarity -- here, the idea relies on the fact that setting a minimum threshold on the similarity that should be considered enables a reduction in the number of node pairs that the similarity have to be computed. The interesting point is that the threshold does not break the relative ranking between similarity scores, as one could expect due to the fact that some nodes are discarded.

Currently, I am investigating the applicability of one or two of the techniques above in a problem that I have in hands now. More about it soon.

References:

[1] http://www.citeulike.org/user/elsantosneto/article/6509946
[2] http://www.citeulike.org/user/ldietz/article/349900

Spam in social software

2009-10-23T08:41:00.000-07:00

From the Greg Linden's blog:

An arms race in spamming social software
by Greg Linden (16 Oct 2009)
http://glinden.blogspot.com/2009/10/arms-race-in-spamming-social-software.html

The post presents an ultra-short, yet interesting, summary of spam strategies observed in social software. In particular, Greg Linden mentions several observed badmouthing strategies -- the spammer tries to taint competitors reputation, as opposed to attempt to use spam to obtain benefits directly. Interestingly, Chen and Friedman [1] conjecture that badmouthing attacks are harder to tackle than attacks that attempt to artificially boost one's reputation.

The post ends suggesting that incentive-based approaches to deter spam are (possibly good) alternatives compared to techniques that detect spam by focusing on content analysis (e.g., detection of commercial intent on blog comments). I would say they are complementary, though.

[1] Chen & Friedman. "Sybilproof Reputation Mechanisms".
http://doi.acm.org/10.1145/1080192.1080202

Relationship between cross-field citations and work impact

2009-08-26T11:25:00.000-07:00

A recent work by Shi, Adamic , Tseng and Clarkson has an interesting analysis on the relationship between works that draw from different areas (i.e., cite papers outside their fields) and their subsequent impact. [1]

One of the interesting bits:

[...]
Intuitively, any individual citation will at most have a very weak impact on the success of a citing paper. It will only be one of possibly dozens of references made in an article or patent. Other factors, such as the publication venue and the reputation of the authors, are more likely to contribute to the impact of the article than any individual citation the authors include. We nevertheless see a significant relationship between the interdisciplinarity of citations and the impact of the publication.
[...]

This reminds me of previous results on the relationship between network constraint and value of ideas [2]. The intuition is that a person who is in a bridge position in her social network (i.e., connecting two distinct groups) is more exposed to different ways of thinking, which may lead to that person having more valuable ideas. Here, the social network is the citation network, and the bridges are papers that cite otherwise unconnected clusters (i.e., fields).

A recipe for higher impact research?

[1] Shi et al. 2009. The Impact of Boundary Spanning Scholarly Publications and Patents. PLoS ONE.
[2] Burt, R., 2003. Structural Holes and Good Ideas. American Journal of Sociology.

The Internet and its Topology

2009-07-15T09:02:00.000-07:00

A while ago, I came across an article that discusses points related to network modeling and characterization [1], particularly the Internet physical topology.

The motivation used by the authors is, as the article puts it, the power-law argument. In particular, the authors highlight important points researchers should focus when performing similar studies. They also go further and challenge the now traditional assumption that the Internet topology resembles a scale free network. In this post, I briefly summarize the paper.

The paper recounts the now well known and accepted scale-free Internet argument (i.e., that the Internet topology is well modeled by a network with a power-law node degree distribution). The arguments, as presented by the authors, are rooted on the limitations of traceroute on accurately determining the physical topology, as it only captures interfaces of routers instead of physical boxes.

In short, the work proposes to focus on the decision making process an Internet service provider goes through when planning and deploying its physical infrastructure, as opposed to using traceroute traces to inspire models for the Internet topology. Moreover, the authors suggest that the right tool to do that is constrained optimization that allows to formalize the referred decision making process.

Other interesting points extracted from the paper:

a).

A node high degree implies low capacity links, conversely low degree implies high capacity links, this is due to the limited capacity on processing traffic. Thus, the high degree nodes would be on the edge of network, as opposed to the core, which is different from what most of the traceroute studies claim.

b).

To avoid confusion and to emphasize the fact that preferential attachment is just one of many other mechanisms that is capable of generating scale-free graphs, we will refer here to the network models proposed in [2].

Suggested principles on characterization and modeling, which, I think, are sufficiently general to be applied in fields other than network topology characterization:

1. Know your data:

The data used by Faloutsos, Faloutsos, and Faloutsos [3] was not intended to capture the physical topology, rather to "get some experimental data on the shape of multicast trees one an actually obtain in the real Internet"

2. When modeling is more than data fitting:

If we wish to increase our confidence in a proposed model, we ought also to ask what new types of measurements are either already available or could be collected and used for validation. (By new they mean completely new types of data not involved whatsoever with the original modelling exercise).

3. Know your statistic:

Once agreeing that the data set is problematic, one could try to use a more robust statistic to avoid the mistakes.

[1] W. Willinger, D. Alderson and C. Doyle. "Mathematics and the Internet: A source of enormous confusion and great potential", Notices of the AMS, Vol. 56, no. 5, May 2009.

[2] A.-L. Barabási and R. Albert, Emergence of scaling in random networks, Science 286 (1999).

[3] M. Faloutsos, P. Faloutsos, and C. Faloutsos, On power-law relationships of the Internet topology, ACM SIGCOMM (1999).

The role of the scientific method

2009-06-23T01:10:00.000-07:00

Here is an interesting article by John Polanyi published by The Globe and Mail.

Hope lies in the scientific method by John Polanyi.

The article discusses the role of science, and in my opinion, more importantly, the importance of the critical view of scientists on the impact that political decisions may have on the lives of many.

Food for thought, indeed.

Data Reliability Tradeoffs

2009-06-03T09:38:00.000-07:00

Abdullah Gharaibeh (my colleague at the NetSysLab) has a recent work that explores a combination of heterogeneous storage components in terms of cost, reliability and throughput.

The work proposes a storage architecture that leverages idle storage resources, located on volatile nodes (e.g., desktops), to provide high throughput access at low cost. Moreover, the system is designed to provide durability with a low throughput durable storage component (e.g., tape).

What I like in the solution is that it shows nicely how to decouple components that provide two important features for data-intensive applications: availability and durability. Moreover, this separation (together with the evidence showed by the experiments) helps system administrators to reason about the deployment cost.

Here is the abstract:

Abdullah Gharaibeh and Matei Ripeanu. Exploring Data Reliability Tradeoffs in Replicated Storage Systems. ACM/IEEE International Symposium on High Performance Distributed Computing (HPDC 2009), Munich, Germany, June 2009.

This paper explores the feasibility of a cost-efficient storage architecture that offers the reliability and access performance characteristics of a high-end system. This architecture exploits two opportunities: First, scavenging idle storage from LAN connected desktops not only offers a low-cost storage space, but also high I/O throughput by aggregating the I/O channels of the participating nodes. Second, the two components of data reliability – durability and availability – can be decoupled to control overall system cost. To capitalize on these opportunities, we integrate two types of components: volatile, scavenged storage and dedicated, yet low-bandwidth durable storage. On the one hand, the durable storage forms a low-cost back-end that enables the system to restore the data the volatile nodes may lose. On the other hand, the volatile nodes provide a high-throughput front-end. While integrating these components has the potential to offer a unique combination of high throughput, low cost, and durability, a number of concerns need to be addressed to architect and correctly provision the system. To this end, we develop analytical- and simulation-based tools to evaluate the impact of system characteristics (e.g., bandwidth limitations on the durable and the volatile nodes) and design choices (e.g., replica placement scheme) on data availability and the associated system costs (e.g., maintenance traffic). Further, we implement and evaluate a prototype of the proposed architecture: namely a GridFTP server that aggregates volatile resources. Our evaluation demonstrates an impressive, up to 800MBps transfer throughput for the new GridFTP service

Individual and Social Behavior in Tagging Systems

2009-04-22T12:31:00.001-07:00

As part of a much broader investigation on the peer production of information, Individual and Social Behavior in Tagging Systems [1] is a recent work that focuses on the quantitative aspects of tag reuse, item re-tagging and the implicit social relation inferred from the similarity of user interests.

The observations point to interesting directions on the design of systems, such as recommendation systems, that aim at exploiting past user activity. For instance, it providers quantitative evidence why item recommendation tends to be less efficient than tag recommendations in these systems (based on the relatively higher level of tag reuse, compared to the item re-tagging).

I must mention that this work is a result of a collaboration with an enthusiastic team: Nazareno Andrade, David Condon, Adriana Iamnitchi and Matei Ripeanu,

Reference:

[1] Elizeu Santos-Neto, David Condon, Nazareno Andrade, Adriana Iamnitchi and Matei Ripeanu. "Individual and Social Behavior in Tagging Systems". In the 20th ACM Conference on Hypertext and Hypermedia. Torino, Italy, June 29-July 1, 2009.

Tracing Influenza Epidemics via Crowdsourcing

2009-03-10T02:04:00.000-07:00

Researchers from Google (Mountain View,CA,USA) and Centers for Disease Control and Prevention (Atlanta, GA,USA) recently reported a method to build a model that helps detecting the spread of Influenza.

The article describes the analysis of search logs in combination with records of doctor visits related to Influenza-like Illness (ILI). The goal of the model is to predict the percentage of doctor visits that are ILI-related. The authors report a correlation of up to 0.96 between the model predictions and the Centers for Disease Control (CDC) reports.

Citation:

Jeremy Ginsberg, Matthew H. Mohebbi, Rajan S. Patel, Lynnette Brammer, Mark S. Smolinski and Larry Brilliant. Detecting influenza epidemics using search engine query data. Nature 457, 1012-1014 (19 February 2009).

One Microsoft Way

2009-03-03T21:52:00.000-08:00

Last week, at Redmond, Microsoft Research showcased a list of projects from its labs located worldwide during the TechFest 2009. As part of my internship last year, at the lovely Microsoft Research Cambridge - UK, I worked on the Mobile Content-Casting project, which was also showcased.

Ars Technica has an article summarizing some of the projects presented at TechFest 2009.

It is worth taking a look: Microsoft Research TechFest 2009: a glance at the road ahead.

I particularly found the SecondLight and the suite of Social-Digital demos really cool. Both are from MS Research at Cambridge - UK.

A nice short story

2009-02-27T12:33:00.000-08:00

This is a fun short story published by Nature that I came across recently.

"Lost in sun and silence -- The golden age of communication"
by Vincenzo Palermo. Nature, Vol 457, 26 February 2009.

The Anticommons

2008-08-13T00:52:00.000-07:00

The New Yorker's Financial Page has an interesting article: The Permission Problem by James Surowiecki (the author of Wisdom of Crowds).

James Surowiecki discusses the notion of anticommons as presented by Professor Michael Heller (The Gridlock Economy).

To illustrate the point, Surowiecki points out two extreme scenarios of the resource sharing problem -- i) common good model: the resource is deemed public and it is shared among individuals without the notion of individual ownership over the shared resource; ii) private property model: the notion of unlimited property, where the resource is owned by a subset of individuals, who may charge other individuals that want to consume units of that resource.

The article says that, on the one hand, common goods may lead to the well-known tragedy of the commons: overuse. On the other hand, unlimited property rights may lead to the exactly opposite: waste of resources (or the Tragedy of the anticommons.

The article has nice examples:

[...]
The commons leads to overuse and destruction; the anticommons leads to underuse and waste. In the cultural sphere, ever tighter restrictions on copyright and fair use limit artists’ abilities to sample and build on older works of art. In biotechnology, the explosion of patenting over the past twenty-five years—particularly efforts to patent things like gene fragments—may be retarding drug development, by making it hard to create a new drug without licensing myriad previous patents. Even divided land ownership can have unforeseen consequences. Wind power, for instance, could reliably supply up to twenty per cent of America’s energy needs—but only if new transmission lines were built, allowing the efficient movement of power from the places where it’s generated to the places where it’s consumed. Don’t count on that happening anytime soon. Most of the land that the grid would pass through is owned by individuals, and nobody wants power lines running through his back yard.
[...]

It seems to me that certain computational environments present an interesting middle ground between these two extremes discussed above. For example, Nazareno pointed out a while ago to a Large-scale commons-based Wi-fi: users, who own an Internet connection, may share the spare capacity in exchange for either using the spare capacity of others later or being paid for it. The wonderful insight of this resource sharing model is that people buy more Internet bandwidth (as many other computational goods -- if I can name it like this) than they are able to use. Hence, resources are mostly underutilized. So, why not sharing it (the spare capacity)in exchange for access to others spare capacity in the future?

Finally, a question comes to my mind: besides the fact that certain resource units bear an extra capacity by definition (e.g. often my CPU is 99% idle), does any other intrinsic resource characteristic play a role in suggesting which model is suitable for the sharing of that resource?

Information Management in Living Organisms

2008-07-27T22:07:00.000-07:00

Nature has an article by Paul Nurse (Life, Logic and Information), where he discusses some ideas on studying living organisms as information management systems.

Paul Nurse suggests that analyzing the information flow in living organisms would help to understand certain behaviors, which are not completely clear nowadays.

From a computer systems researcher standpoint, the interesting aspect of Paul Nurse's idea is that it goes on the opposite direction of previous studies: instead of drawing inspiration from nature to build information management systems (e.g. ant inspired algorithms), the author proposes to use information science tools to study the nature.

A piece from the article:

Systems analyses of living organisms have used a variety of biochemical and genetic interaction traps with the emphasis on identifying the components and describing how these interact with each other. These approaches are essential but need to be supplemented by more investigation into how living systems gather, process, store and use information, as was emphasized at the birth of molecular biology.

This sounds exciting, as a better understanding of living systems could feedback into the bio-inspired approach of designing distributed computational systems.

A couple of years ago, I briefly explored the design of a distributed storage system based on the behavior of the Messor Barbarus ants (for more details on the M. Barbarus ants see Anderson, C., J.J. Boomsma, and J.J. Bartholdi, III. 2002. Task partitioning in insect societies: bucket brigades. Insectes sociaux 49(2): 171-180).

The rationale behind it is quite simple: every time an unloaded larger ant encounters a loaded smaller ant, the load is passed from the smaller to the larger ant. This naturally spread the work among the workers according to their capacity (strength and speed).

Bringing it back to the context of distributed storage systems, the idea is to enable a self-organizing load balance scheme by making larger nodes to request more load from lower capacity nodes. The goal is to improve throughput and data availability.

Obviously, a comprehensive performance evaluation is necessary to claim that this strategy would lead to an globally efficient system.

HPDC'08 - Part II

2008-07-03T10:44:00.000-07:00

Gilles Fedak delivered an invited talk at the UPGRADE-CN workshop (part of the HPDC'08). He presented the BitDew - a programmable environment that targets data management in Desktop Grids [1].

The rationale behind BitDew is that applications can define routines for data manipulation. These routines are expressed via predefined metadata, which are used by the infrastructure mechanisms to perform data management tasks, such as replication.

In a Technical Report, Gilles and colleagues present use cases and performance evaluation of mechanisms that provide data management functionality in BitDew.

In particular, I found the approach of leveraging metadata interesting. The predefined set of metadata allows the application layer to communicate requirements to the infrastructure regarding the desired level of fault tolerance and transfer protocols, for example.

In fact, this intersects with one of our projects at the NetSys Lab, where we investigate the use of custom metadata as a cross-layer communication method for storage systems [2].

As we use the file system interface to separate between the application and the storage layers, the two approaches (BitDew and our Custom Metadata approach) seem complementary. The metadata passed by the applications via BitDew could propagate to the file system, where it would interact with the Extended Attributes interface (which is exploited by our solution).

More coding fun to come...

[1] Fedak et al. "BitDew: A Programmable Environment for Large-Scale Data Management and Distribution". Technical Report, 6427, INRIA.

[2] Elizeu Santos-Neto, Samer Al-Kiswany, Nazareno Andrade, Sathish Gopalakrishnan and Matei Ripeanu. "Enabling Cross-Layer Optimizations in Storage Systems with Custom Metadata". In HPDC'08 - HotTopics. Boston, MA, USA. September, 2008.

HPDC'2008 - Part I

2008-06-30T12:20:00.000-07:00

Last week I participated to two great conferences: the International Symposium in High Performance Distributed Computing (HPDC) and the USENIX. Both events took place in Boston, MA, USA.

There a lot of interesting things to mention. Thus, to avoid a single long post, I will describe a few presentations that I attended (and discuss some ideas) in a series of short posts.

In the first two days at HPDC, there were two interesting workshops: UPGRADE-CN (P2P and Grids for the Development of Content Networks) and MMS (Managed Multicore Systems).

The two workshops had works related to the research projects I am currently working on.

Molina [1] presented his work on designing two protocols that enable collaborative content delivery in mobile transient networks. By transient networks, the authors mean networks composed of devices that are geographically co-located for a short period such as a music festival.

The authors suggest to exploit the multiple network interfaces currently available in most mobile devices and to enable collaborative use of these multihomed devices.

The idea is quite interesting. In particular, it raises some issues from the perspective of distributed resource sharing. It would be good to understand whether incentive mechanisms are necessary in transient networks. The idea is to encourage users to share their connections with a community for collaborative downloading/streaming of content.

On top of that, a nice follow-up work would be to investigate the feasibility of collaborative data dissemination protocols, which are widely used in the Internet (e.g. BitTorrent), in the transient networks scenarios.

[1] Molina et al. "A Social Framework for Content Distribution in Mobile Transient Networks". In UPGRADE-CN'2008.

IPTV Viewing Habits and Netflix Player

2008-06-05T10:11:00.000-07:00

The IPTPS 2008 has an interesting paper on exploiting TV Viewing Habits to reduce the traffic on the ISP backbone generated by IPTV consumers.

"On Next-Generation Telco-Managed P2P TV Architectures" by Meeyoung Cha (KAIST), Pablo Rodriguez (Telefonica Research, Barcelona), Sue Moon (KAIST), and Jon Crowcroft (University of Cambridge).

In this paper the authors analyze the utilization o P2P content distribution techniques to reduce network overhead in a Internet Service Provider IPTV infrastructure. To exploit the patterns of channel holding time, channel popularity and the correlation between the time of the day and the number of viewers, the authors propose a locality-aware P2P content distribution scheme that reduces the traffic on the ISP backbone.

From the paper:

we ascertain the sweet spots and the overheads of server-based unicast, multicast, and serverless P2P and also show the empirical lower bound network cost of P2P (where cost reduction is up to 83% compared to current IP multicast distribution

[...]

We believe that our work provides valuable insights to service providers in designing the next-generation IPTV architecture. Especially, it highlights that dedicated multicast is useful for few of the extremely popular channels and that P2P can handle a much larger number of channels while imposing very little demand for infrastructure.

This week I saw some news on the internal characteristics of the Netflix Player. Immediately, I thought of the paper from IPTPS as a possible optimization to the Netflix Player.

The NetFlix player is supposed to use the conventional broadband connection, as opposed to well provisioned IPTV architectures described in Cha et al. Perhaps, the locality-aware P2P content distribution technique is even more interesting in the Netflix Player case.

Nevertheless, the viewing habits and interest sharing among Netflix users may differ dramatically from what is observed in the IPTV environment, which would impact the efficiency of the locality-aware P2P content distribution.

Oh My Goosh!

2008-06-03T07:54:00.000-07:00

If you like typing your commands away to interact with your computer, you will like this: Goosh :-)

From Slashdot: goosh, the Unofficial Google Shell posted by kdawson on Monday June 02, @07:26PM.

It's essentially a browser-oriented, shell-like interface that allows you to quickly search Google (and images and news) and Wikipedia and get information in a text-only format.

"Yes, There Is a Correlation"

2008-05-28T10:46:00.000-07:00

This week I came across an interesting paper: "Yes, There is a Correlation - From Social Networks to Personal Behavior on the Web" by Parag Singla (University of Washington) and Matthew Richardson (Microsoft Research) in WWW'2008.

In summary, they show that the similarity between the personal interests and attributes of two users who are MSN contacts is much higher than two random users. Moreover, I've found the problem formulation elegant and the scale of data non-trivial to handle (approx. 13 million unique users).

From the paper:

Summarizing the results, we showed that people who talk to each other on the messenger network are more likely to be similar than a random pair of users, where similarity is measured in terms of matching on attributes such as queries issued, query categories, age, zip and gender. Further, this similarity increases with increasing talk time. The similarities tend to decrease with increasing average time spent per message. Also, we showed that even within the same demographics, people who talk to each other are more likely to be similar. Finally, as we hop away in the messenger network, the similarity still exists, though it is reduced.

I wonder whether a similar level of correlation would be observed in online communities with other purposes, such as content-sharing (e.g. Flickr and YouTube).

OurGrid 4.0 released

2008-05-16T02:09:00.000-07:00

Good news from the South! The OurGrid 4.0 is out.

In summary: OurGrid is an open source, free-to-join, peer-to-peer grid, where users trade computational resources. The loosely coupled computational infrastructure is ideal for the execution of embarrassingly parallel applications.

I am particularly glad with this release, as OurGrid has been a useful tool in my previous studies. I used it to analyze traces of activity of content sharing communities. OurGrid makes it easy to harness the idle times of our desktop machines and monitor the progress of the computations in a much easier way.

Next week I will definitely give a try on the new version (as we are still using version 3.3).

The new site looks great too. Congratulations, OurGrid Community! :-)

Deep Water(*) at 45rpm

2008-05-15T13:07:00.000-07:00

A bit about recent music experiences...

It has been a while, since I listened to the voice of Beth Gibbons. So, I recently indulged myself with the addition of a new record to my collection. The Portishead new album: Third. By record, I mean an LP. :-)

The immediate surprise after playing the first song was to hear some voices in Portuguese. For a moment, I thought that the record player switch was set to FM Radio on some independent radio station.

The second big surprise was that record sound dark, with very slow beats and bass sounds. After a couple of seconds, I noticed that the album is an 45 RPM LP, as opposed to the usual 33 RPM records (more popular today). A quick adjustment on the player rotation speed and voilá!.

Overall, Portishead's new album sounds quite different from the previous works. I feel that there is more emphasis on the drums, which is really nice. Beth Gibbons voice is more discrete than in her solo album or in the previous Portishead recordings.

In my opinion, the best song of the album so far is Magic Doors (side 4, as the package comes with 2 LPs). :-)

Unfortunately, I missed their Coachella's concert, but I hope to have another opportunity soon...

(*) Deep Water is one of the songs of the album. I think it would work as a nice title for it.

Non-absolute numbers

2008-04-27T18:51:00.001-07:00

As you come back to life (after a series of conference and project deadlines), you are able to recover clever and funny pieces of literature such as the explanation of what is Bistromathics and its relation to the non-absolute numbers.

Making faces in virtual worlds

2008-03-05T12:17:00.000-08:00

Today, a friend sent me a link to this device named: Emotiv.

A rough description of it would be: a helmet that monitors your brain activity and convert it into signals that can be used in a variety of ways.

Although games might be the first thought, the Expressive application particularly reminded me of a previous post by Ian Foster on a Second Life hack that allows the manipulation of objects in virtual worlds as a response to some physical process.

Hadoop - now in larger scales!

2008-02-21T20:28:00.001-08:00

Yahoo! just reported their new deployment of a Hadoop-based application. The achievement is considered to be the world's largest Hadoop deployment in a production environment.

The scale of the application is quite impressive. They used Hadoop to process the Webmap, as part of their search engine architecture. From their post (check their website for a video with some discussion about Hadoop in this context):

* Number of links between pages in the index: roughly 1 trillion links
* Size of output: over 300 TB, compressed!
* Number of cores used to run a single Map-Reduce job: over 10,000
* Raw disk used in the production cluster: over 5 Petabytes

I can even see the difference in the quality of the search results now. :-)

Update: Greg Linden also posted about the new Hadoop-cluster. It's nice that he puts the numbers above in perspective, by comparing to Google's infrastructure.

Interesting Articles: IPTPS 2008

2008-02-12T17:20:00.000-08:00

For those interested in the convergence of Online Social Networks and Peer-to-Peer Systems, it is worth taking a look at some articles in the program of the International workshop on Peer-To-Peer Systems (IPTPS).

Taking photography to new heights

2008-01-29T19:21:00.000-08:00

In 1906, some panoramic pictures of San Francisco after the big earthquake were taken. These were not ordinary pictures. George Lawrence used kites to place a camera at the right place to record the extension of the damage caused by the earthquake. Besides the historic value of the pictures, they are the outcome of a quite interesting engineering project.

Two years ago, Lawrence's project was revisited. Although they did not use kites this time, the picture is still impressive. They also have some interactive version that allows you to zoom in and see more details of the landscape.

Scientific Data For All!

2008-01-19T15:36:00.000-08:00

The Wired Blog is running a brief article about yet another Google's initiative. The idea is to provide storage, and as far as I understood, free access to scientific data sets.

One interesting point of the article is the following:

(Google people) are providing a 3TB drive array (Linux RAID5). The array is provided in "suitcase" and shipped to anyone who wants to send they data to Google. Anyone interested gives Google the file tree, and they SLURP the data off the drive. I believe they can extend this to a larger array (my memory says 20TB).

It sounds exciting that in the near feature, we might have access to a long list of data sets. Perhaps, under a standard API. If you like buzzwords, this might be named (if it is not the case already) -- Science in the Cloud. Despite the name they will give to this, this initiative can bring a long list of advantages for the the scientific community, I think.

Finally, I wonder when the RFC for the "suitcase-based transport protocol" will be available (similar to RFC 1149). :-)

2007-12-18T14:24:00.000-08:00

Content Sharing with Good Privacy Control

This is a follow up on the previous post.

Open Tag is a tool where users do have the option to select what is private and to which degree. Moreover, users may withdraw completely from the system by removing their activity traces.

2007-12-12T17:41:00.000-08:00

Privacy is on the news!

These days, online privacy have attracted a great deal of attention in technology media [3][4]. Perhaps, the online privacy topic is an old concern in some specific conversation/technology/academic circles, but privacy control was apparently dragged to the general public attention more recently due to the surge of online social networks.

It was interesting to see that online service providers as Ask.com is try to sell privacy control as one feature that differentiates them from their competitors [1].

Although there is an interest in the market to raise some awareness about privacy and to allow users to control the access to their online footprints [2], there is a question whether this will attract more consumers or not.

Regardless, I think the user must have the option of fine tunning the disclosure of his/her explicit and implicit online "footprints". Moreover, I believe that certain domains could benefit from systems designed around the high-level concept of online social networking, if better privacy control capabilities are put in place.

References

[1] Ask Eraser. http://sp.ask.com/en/docs/about/askeraser.shtml
[2] Attention Trust. http://www.attentiontrust.org.
[3] Will Privacy Sell?. Slashdot.org. December, 11, 2007.
[4] Evolving Privacy Concerns. MIT Technology Review. December, 11, 2007.

2007-11-03T16:49:00.001-07:00

GPS-2-GPS

No need for one more drop in the bucket of news about Google Social API.

Instead, I will comment on a kind of old, but interesting, news.

An article by Roy Furchgott (Navigating With Feedback From Fellow Drivers) in the New York Times describes a new GPS-enabled device that is able to receive traffic information based on the aggregation of the information collected from other cars using the same device.

The rationale behind the Dash Express (as the device is named) is that there is a wealth of information that individual cars can generate. Moreover, if such information is aggregated it can become extremely useful for drivers. Examples are obviously related to traffic, but they can also be related to business around a certain area (such as parking space).

The idea is neat. However, the aggregation is currently done in a centralized fashion. I do not have a clear picture on whether this centralized component limits the scalability or not.

In any case, there seems to have an interesting research problem in enabling a Dash-2-Dash communication, without relying on a central point to aggregate the information, and yet maintaining the quality of information by coping with devices that could report wrong data (maliciously or due to mechanism faults). In fact, there are some interesting results in the community of wireless sensor networks along these lines.

2007-10-06T01:12:00.000-07:00

Internet OS

Today, Xcerion contacted me. Just to let me know that I was not selected in the first batch to receive a developer/beta tester of their Internet OS - XIOS/3.

Although the site contains more information than a couple of months ago, as expected, their description of the underlying technologies are opaque.

It seems that that I'll have to wait a bit more...

2007-09-15T22:24:00.000-07:00

On The Repeatability of Experiments

As someone with an experimental background, I have found this initiative by the organizers of PODS/SIGMOD quite interesting.

In their page about guidelines for research papers, the program committee not only present the format guideline, but also a set of steps to help the committee to assess the "repeatability level" of an experimental research.

From their page:


...
we attempt to establish that the code developed by the 
authors exists, runs correctly on well-defined inputs, 
and performs in a manner compatible with that presented 
in the paper.
...

2007-08-31T18:26:00.000-07:00

respice finem

These days, a passage from a book I read a while ago reoccurred to me several times. It sounds interesting and it seems to have a broad applicability in science (and in life, if you wish).

[...]

Look at the unknown. This is old advice; the corresponding 
Latin saying is: "respice finem." That is, look at the end. 
Remember your aim. Do not forget your goal. Think of what is 
required. Keep in mind what you are working for. Look at the
unknown. Look at the conclusion. The last two versions of 
"respice finem" are specially adapted to mathematical problems,
to "problems to find" and to "problem to prove" respectively.

[...]

From: G. Polya. "How to Solve It: A New Aspect of Mathematical Method", page 123, 2nd Edition, 1973.

The only addendum, perhaps, is that the goal may change time to time.

2007-07-21T14:31:00.000-07:00

Open Books!

Rainy day. A cup of Ethiopia Harar and a book.

Today, while drinking a good cup of coffee, I came across the Open Library Project. At first, I thought the book reader used by the Open Library Project was the GreenStone (a tool developed by a digital library project in New Zealand at the University of Waikato with a long list of collaborators). As a matter of fact, it is not.

It is good to see that there are a lot of initiatives out there aiming to provide people with open access to books from a whole variety of topics. Projects like Project Gutenberg and Open Library are good examples that these efforts are up and running.

Seeing digital books as a component in the content generation/consumption ecosystem, it would be interesting to see the convergence of open libraries and urban-driven search tools. A good sample is the Google Books that allow readers to find the closest library that has a particular title available.

Perhaps, the next generation of such systems will also suggest what is the coffee shop that serves your favorite coffee to accompany the book you are searching for. :-)

2007-07-01T11:51:00.000-07:00

Happy 1st of July

For many reasons, my interest over virtual worlds is increasing everyday. Currently, the attention relies on the characterization of large scale distributed systems perspective, than from a business perspective (which is may be very interesting too!).

In the Scientific American's blog, Christopher Mins comments on a talk delivered by Mitch Kapor (among a list of activities, he is also investor and chair of Second Life).

The most interesting aspect of this talk, which seems natural to me, is the suggestion that Kapor (also as a chair of the Mozilla Foundation) defends the creation of open standards for virtual worlds.

It would be cool to think of a protocol design to allow interoperability among virtual worlds. Although this seems to be a classical software engineering problem, the requirements posed by virtual worlds (including the demand for monetary transactions) may be unique enough to create new research opportunities.

By the way, happy 1st of July. :-)

2007-06-24T17:35:00.001-07:00

Digital Libraries and User Attention

This past week, I attended to the JCDL'2007 (Joint Conference in Digital Libraries) and the CAMA'2007 (International ACM/IEEE Workshop on Contextualized Attention Metadata).

Since I cannot comment every single interesting paper that I've seen and discussed about (there are so many), I will point two interesting papers:

The first, which was also the very first presentation of the conference, World Explorer: Visualizing Aggregate Data from Unstructured Text in Geo-Referenced Collections (presented by Rahul Nair). This is a cool tool built on top of Flickr geotagging features. It is really nice to see how many applications are possible considering on-line communities like Flickr and del.icio.us that incorporate tagging features.

The main opportunity explored by the authors is to use geo-reference to cluster content. Besides a simply photo sharing mechanism, I think geotagging also opens up several research challenges/opportunities on designing applications for urban sensing.

The second paper was Can Social Bookmarking Enhance Search in the Web? (presented by Y. Yanbe).

The authors propose the introduction of what I would call a "Ranking Aggreagator" mechanism between the user, Google PageRank and del.icio.us ranking. Thus, their observation is that extremely fresh web pages tend to get low PageRank, but they may have a fair number of bookmark occurrences in del.icio.us. Therefore, they propose a combination of both ranking schemes to improve the ranking of fresh pages and allow the user to get good mix of 'reputable' pages via PageRank and popular pages on del.icio.us. Actually, I wondered if a combination of Google search history and the user interest sharing could be combined to provide better personalized search results.

I also participated to an interesting workshop, CAMA'2007, organized by Erik Duval, Martin Wolpers and Jehad Najjar.

The first talk by Seth Goldstein was exciting, possibly because it shows the incredibly large number of business opportunities are orbiting on-line social networks and how much value there is on online users attention

Joe Pagano from the Library of Congress presented some results on measuring the audience of a newly launched web site. His main finding was that more visitors come from blogs than from search engines, just to reinforce the intuition on the blog influence on the information consumption in the Web.

Personally, a positive aspect of this workshop was to identify possible applications that may validate our preliminary studies on interest sharing in collaborative tagging communities. For example, an extension on Joe Pagano's work would be the application of our interest sharing graph to understand how these visitors relate to each other and whether they from sub-communities of interest.

Also, Erik Duval made a nice comment on the fact that the large number of unique users we have found in our investigation over CiteULike and Bibsonomy may still present rich information to recommendation systems, even though these users are not connected to any island of interest we depicted in the interest sharing graph (more details here).

This is a brief summary of what I have seen this past week. Now, a lot of ideas to refine and put in practice...

2007-06-04T08:23:00.000-07:00

Teragrid, Group Theory, Rubik's Cube

The Teragrid has been used to help finding the solution for the minimum number of steps that lead to the solution of the Rubik's Cube.

What impressed me more was the reduction of the initial number of possible states to enable the finding. Very neat!

See more here: How many moves does it take to solve a Rubik's Cube?
By Matt Ford

2007-06-03T19:18:00.000-07:00

Issues on Online Reputation

The Information Week has an interesting article on the issues involving the proliferation of web communities which are built around user generated content and mechanisms to build user reputation.

Web Credibility: Hard Earned, Harder To Prove
by J. Nicholas Hoover

2007-05-25T12:40:00.000-07:00

Problem Solving Skills

Some works by George Polya captured my attention recently. In part, this happened because some of his works provide very powerful models to characterize large scale content/resource sharing communities (or systems, if you wish). For example, the Urn Model is suggested by Golder and Hubberman as a model to explain collaborative tagging behavior.

Another interesting work, which I am reading now, is the book How to Solve It. This book brings a methodology to motivate students on the process of strengthening their problem solving skills. I wonder how different it would be if some undergrad professors inspired a little bit of their course mechanics on that material.

2007-04-24T14:59:00.000-07:00

Aerotrio

Today, after a random walk on the web graph, I hit the website of band that like a lot: Aerotrio.

A recommendation is: while listening carefully to their music, appreciate some photographies by Salvatore. At least, it seems a good combination to me.

It would be great to see a concert by Aerotrio someday again.

2007-03-22T22:43:00.000-07:00

News from the Distributed Storage World

There was a post running on the FUSE mailing list that captured my attention this week. A Google FS-like sort of distributed file system named Startfish.

In their FAQ, there are several claims (and bar charts) about good performance results regarding several performance metrics (i.e. read/write throughput, scalability, etc) compared to NFS, SAMBA and Lustre file systems. But, I think a deep evaluation seems to be needed to assess these claimed advantages.

Also, this seems to be a major milestone on the increasing number of FUSE-based file systems. It would be nice to try it out.

2007-03-14T20:48:00.000-07:00

The Art of Tagging Libraries

It turns out that there is a system that allows users to tag virtual library catalogs.

Thanks to my fellows at OpenTag.

The system is called PennTags and it is developed by people at University of Pennsylvania.

2007-03-08T21:49:00.000-08:00

The Art of Tagging

This seems a very interesting idea on how to extract the perception of public about objects, particularly, art works. Furthermore, it is also a clever way to understand what is the vocabulary actually used by the majority of the audience to describe the art works.

The Art Museum Social Tagging Project

There is a workshop paper about it in the program of WWW2006 Conference:

Investigating social tagging and folksonomy in art museums with steve.museum

An interesting application of the same idea would be to add tagging capabilities to online library catalogs, where users of public libraries could categorize items that they've read (or intend to) by using tags online. This could give a clue about the content of the item, a kind of highly compacted review.

2007-03-02T17:47:00.000-08:00

Rethinking your role...

The BBC has published this article about a petition by European institutions requesting a that all government funded research should be easily available to the public - you can read the article here http://news.bbc.co.uk/2/hi/technology/6404429.stm.

Particularly, the idea of providing open access to government funded research publications seems a sensible move. However, the whole picture is a bit more complex, I guess. The article nicely points out what it may be the fundamental question regarding the discussion on "To Open or Not To Open", which is the role of researchers in disseminating information.

2007-02-16T19:44:00.000-08:00

"High Fidelity"

A short break on popularity distributions and clustering coefficients.

The novel High Fidelity by Nick Hornby is guaranteed fun. The movie based on the novel, which is located at Chicago, is also great. I also must confess that I got the Top Five fever for a while after reading the book and watching the movie.

Thus, just for your fun consideration, appreciated reader, here is my top 5 ranking for cover versions of songs.

1. "Garota de Ipanema" by RUSH (originally by Tom Jobim and Vinicius de Moraes)
2. "What a wonderful world" by Joe Ramone (orginally by Louis Armstrong)
3. "Have a Cigar" performed by Foo Fighters (originally by Pink Floyd)
4. "Bullet in the Blue Sky" by Sepultura (originally by U2)
5. "Hurt" by Johnny Cash (originally by NIN)

Now, a nice place to find those songs... Hype Machine, a nice tool for search music and video content commented on blogs.

Enjoy,
Elizeu

2007-02-03T14:05:00.000-08:00

Hemingway and FLP

Yesterday, I have finished reading For Whom the Bell Tolls by Ernest Hemingway. It is not only because the book is written by a Nobel Prize winner that it is worth reading, it is a very interesting and thought provoking book about the human nature. My impression is that the book becomes more and more exciting as it comes to the end.

Anyway, the main goal of this post is not solely discuss my impressions about this Hemingway's book. The fact is that for several chapters the main character faces a dilemma, which is in my opinion the same challenge faced by those building reliable distributed systems, this is explained by a great paper well known as - FLP. :-)

(Un)fortunately, Robert Jordan could not have access, at that time, to the ground breaking paper by Fisher, Lynch and Patterson (Impossibility of distributed consensus with one faulty process) to shed some light (or darkness, perhaps) on his situation.

Definitely, both works are great references...

2007-01-21T18:16:00.000-08:00

The Power of Probability

An article published at Live Science defends the idea that it is possible that two snowflakes are alike. At least if you believe in probability.

The article says that it is possible to have two snowflakes alike, but with very low probability. Moreover, it totally depends on the way similarity is defined ( number of crystals and shape, for example).

So, be aware when you use the setence "no two snowflakes are alike".

For those who like macro photography, the pictures are very nice.

2007-01-10T23:16:00.000-08:00

Zenit 12XP

A bit of photography now.

As a result of a complex conjunction events, I decided to use my first reflex camera again for a couple of weeks.

It might sound odd, but there is a kind of unique experience on handling a totally mechanic camera as opposed to a modern one (film or digital). From the film loading process, which is more like a ritual than a technical process, to the feeling-based exposure settings.

I've got very nice experimental results with this came in low light conditions. However, what impressed more was color saturation levels I got in a set of pictures.

There was an expedition where I took the same pictures with the Zenit 12XP and an EOS 300 (EOS Rebel in USA) with the same negative film, developed at the same place.

I was planning a set of photographs of urban landscapes with snow. Unfortunately, the shutter mechanism presented some problems in the first frames of a film last weekend. Incredible sunset light tones illuminating the buildings on my neighborhood, but the camera asked for a retirement.

Great times we had together!

2006-12-13T16:51:00.000-08:00

BloggerFS

Last night I was playing around with FUSE, FUSE-J and Blogger Data API. The result is a new FUSE-based filesystem, which is named BloggerFS.

The idea is very simple, by using BloggerFS one can manipulate posts on her blog as if they were files. The user just need to pass a mounting point, a blog address (it needs to be on Blogger) and a pair username/password.

I intend to publish the source code under GPL as soon as the code is fairly documented and ready for use in production. It would be nice to hear your experiences when using it.

Cheers,
./Eli

2006-12-13T06:05:00.000-08:00

Netherlands goes all digital

The government completely siwtched the analog television transmission to digital. The main reason was what surprised me: only 74,000 households will be directly affected by this shift.

According to the news, cable-TV represents 94% of the TV market in Netherlands. This is pointed as one more reason to increase the competition by expanding the coverage of digital TV.

To be able to receive the digital TV signal, users will have to buy a tuner which is estimated to cost around $66.50. In terms of costs, this seems to be much cheaper when comparing to the costs per household to maintain the "free" analog TV. The governement will save $200 per household (a year) with this shift according to the information published at Chron.com (link below).

It seems to me very interesting the role of people that need to think about new formats for TV shows, movies "made for TV", news, and so on, assuming a new technology to transmit them. I had a very inspiring converstion about how Video Art has changed the way people do TV shows, and way people think of doing a TV show is completely different fom cinema. Interestingly, a decisive aspect is the technology that is going to be used for recording and transmitting.

Source: http://www.chron.com/disp/story.mpl/headline/world/4393351.html

2006-11-27T11:32:00.000-08:00

First Impressions of Supercomputing & VTCD'06

I guess this is a late post. However, I still think it is relevant to say some words about my impressions of Supercomputing Conference 2006 (SC'06).

The SC'06 was held in Tampa, FL this year. As my first time in the conference, I had a bit of expectation about how large the conference is. It was very good to see the mix of industry and academia there. The main challange maybe is to measure how the demand posed by industrial scale clients would be driving research labs and university efforts.

The First International Workshop on Virtualization Technology in Distributed Computing (VTDC'06) was held in conjunction with SC'06 and it was a great opportunity to meet researchers involved in virtualization technologies.

I have been most of the time attending to the technical sessions, where there were a lot of interesting works and conversations. Here is a small list of papers that I have found neat ideas and important findings:

Sage A Weil et al. "CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data"

Zheng Zhang et al. "CycleMeter: Detecting Fraudulent Peers in Internet Cycle Sharing"

Sotomayor et al. " Overhead Matters: A Model for Virtual Resource Management". VTDC'06.

David Wolinsky et al. "On the Design of Virtual Machine Sandboxes for Distributed Computing in WOWs"

Hopefully see you on SC'07...

Cheers,
./Eli

2006-10-19T05:35:00.000-07:00

Citizen Collaborative Work

Recently, I have been reading some articles on policies and mechanisms for collaborative work on virtual environments like the Web and I have found some interesting works on reputation schemes to ameliorate the quality of collaboration on collaborative content production.

A while ago I came across an article by Tom Cross on First Monday (Puppy smoothies: Improving the reliability of open, collaborative wikis) which proposes a simple mechanism to allow users to identify parts of a Wiki article that it is not "mature" yet. The rationale behind the approach proposed by Cross is that texts that survive a sufficiently long period of iterative and collaborative editing process may be considered mature and accurate. Conversely, texts recently added may contain inaccuracies. So, readers should be aware of which part is still not mature.

Today I have found a site which promotes the collaborative content production, Bikely. The idea is to allow users to share their bike routes, to add comments and share information about these routes. Basically, a use case for a combination of Wiki and Google Maps. :-)

In that context, I was wondering about a mechanism which could improve the quality of information about routes and comments on routes. In my first thought, the assumption used by Cross may not apply here, since routes could be out of date over time in contrast to text that gets mature. It has also the case where routes and comments are misleading. Anyway, I believe this is a good motivation to think about a reputation scheme with a nice application.

By the way, check out my routes:

- Flamingo - UFCG (Campina Grande, PB, Brazil).
- Chesnut Park - UC Davis (Davis, CA, US)

Cheers,
Eli

2006-09-13T14:54:00.000-07:00

"I read banned books"

I have seen a new post on the The Google Blog about a list of banned and challenged books from 20th century. It was nice to see that a lot of my favorite books are part of the list of best novels of the 20th century have been challenged or banned.

The interesting fact on that is that I do not usually decide about reading a book by magazine reviews and/or ranking of latest week's best sellers. I'd rather to collect good opinions in person talking to my friends. However, most of the time I discover good authors by 'accident' (by watching movies that refer some particular authors or by listening to some music which is inspired in some books). :-)

Furthermore, the list contains several books that I have already put on my shelf. Now, I was wondering what would be banned today ?

Keep reading! ;-)
./Eli

2006-09-05T15:11:00.000-07:00

Fall is coming... Not really!

Alright, the Fall is almost coming, right? Not really. :-) I am in Brazil now and here Spring starts late this month. Thus, the inspiring summer days in Ryerson Hall at University of Chicago working in collaboration with Kate Keahey from Argonne National Labs as part of an internship are behind now.

However, we will be able to remember our wonderful conversation about Shakespeare and Tom Stoppard plays, since our paper was accepted in the Supercomputing'06. The work is entitled To Bid or Not To Bid: A Hybrid Market-Based Resource Allocation Framework. Actually, talking about those authors was motivated by the title that I suggested to the paper. I almost included 'Arcadia' and 'Romeo & Juliet' in the references. :-)

Now, I am part of the LSD-family (Laboratorio de Sistemas Distribuidos) again, at least for the next 4 months. The idea is to be involved with the same research topics that I have been looking into during the summer season at Chicago, but the application is different. The goals is to improve the virtualization solution provided by the OurGrid. I am very glad to be here after one year away to meet my former advisors and friends.

So, I'm looking forward to be in the Supercomputing in November. I hope to see most of my friends from Chicago there. Making new friends is part of the plan as well. :-)

Cheers,
Eli

2006-08-08T21:24:00.000-07:00

Lollapalooza Days

This might seem off topic. However, I would like to comment on my experience at Lollapalooza.

I had the great opportunity to go to Lollapalooza last weekend. The festival was located in Grant Park at Downtown Chicago. I long list of bands played during three days of a good (sometimes, not so good) rhythm mixture.

Lollapalooza was probably the biggest music festival that I have been so far. I would say that the second in size was the Rock In Rio III (Rio de Janeiro, Brazil). The infrastructure of Lollapalooza was perfect, my only complaint was the fact that they were not selling beer, only Budweiser. :-) The concerts were perfectly scheduled and I really enjoyed the whole environment around Grant Park and Downtown.

Last Monday, a friend sent an article from MIT Technology Review about Technology @ Lollapalooza. This is just one more example that networked technology is part of our daily lives.

Despite the fact that the technology device that I took more advantage was a portable fan with an attached water mister during the very very hot weather on Friday, it was interesting to see the high-tech tends packed.

I was wondering later that one could enjoy this huge live crowd to perform a kind of field experiment with sensor devices attached to people. Perhaps, some of the collected data in this "live experiment" could be useful for future simulation based studies.

2006-08-01T19:17:00.000-07:00

The Backup Power

The fact that backup is an important part of any IT infrastructure management plan, everybody should know. What about backing up DNAs?

Well, this is what a NYTimes article by By Richard Morgan ("Life After Earth: Imagining Survival Beyond This Terra Firma") suggests (in a scientific fiction mode) that it could be done to prevent human life of being extinct by natural or war disasters.

Despite the currently unrealistic, yet possible, interesting description of the applicability of a DNA repository of every form of life in our beloved planet earth, the article discuss how the doomsday is understated sometimes.

However, instead of putting all effort on developing technologies for "DNA backup" and storage, how about to move the focus from military development to sustainable development? I guess this would be a very good and true step to reduce the risk of human extinction.

Anyway, reading this article was good to reassure my point of view that some priorities of some nations do not seem to me the right way to go. :-)

Cheers,
./Eli

2006-06-17T17:35:00.000-07:00

Greenwood Simulator

It's been a long time, since the last time I've posted. Alright, I guess I'll have more time for the rest of the summer to post some interesting thought about what really matters in life: People, Music, Photography and Computers. :-)

The good news is that I have started a new solo project last Spring. After having a laptop crash I decided to publish the partially recovered source code that I have been developing under the terms of an open source initiative.

The main goal is to develop a simulator (or something more like a simulation framework) to help researchers interested in having fun with data staging on large scale and distributed storage environments.

GreenSim builds on top of Simgrid Toolkit. My main goal is to provide a developer-oriented framework that can be personalized to specific demands.

However, before downloading you should be aware that the software is still in a pre-alpha version. Thus, you should not expect a stable version. I will keep committing changes, bug fixes and announcing releases here, as soon as the internship activities allow me to do.

In the case you feel interested in participate, please, drop me a message and let's discuss your ideas, I would appreciate it!

GreenSim
http://greensim.sourceforge.net

Regards,
./Eli

2006-05-26T18:59:00.000-07:00

Is Computer Science a science?

Time to time this question comes to my mind, and (almost automatically) I remember a statement by one of the most interesting persons that I have met in Chicago while we were enjoying some beers with our advisor:

"Real sciences do not need to have the word science in their names. For example, Physics, Biology and Chemistry."

I should start this post by saying that I disagree with such statement. :-)

Let us take a look at what is the definition of the word science According to the Merrian-Webster Online (http://www.m-w.com). Thus, we have:

1: the state of knowing : knowledge as distinguished from ignorance or misunderstanding

2 a : a department of systematized knowledge as an object of study (the science of theology) b : something (as a sport or technique) that may be studied or learned like systematized knowledge (have it down to a science)

3 a : knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method b : such knowledge or such a system of knowledge concerned with the physical world and its phenomena : NATURAL SCIENCE

4 : a system or method reconciling practical ends with scientific laws (culinary science)

To avoid being biased and a situation where one could tell me that I do not know the color of the sky. I will cite a second source. No, the second source is not Britanica, it is Wikipedia (http://en.wikipedia.org/wiki/Science).

[...]
refers to the system of acquiring knowledge based on empiricism, experimentation, and methodological naturalism.
[...]

Therefore, I think that Computer Science fits very well on both definitions and it should be considered a real science. Perhaps a good next question is: what is a good scientific method in computer science and what is not?

However, this is a discussion for a second round of beers. :-)

Cheers,
Eli

2006-05-19T23:10:00.000-07:00

The Spam World Map

I have found an interesting tool based on the Google Maps API and Host IP Info API. The tool basically translates domain names to a geographic location by showing on the map. The idea is pretty simple. But it is nice. :-)

Here is the link: http://map.butterfat.net/emailroutemap/

It was a very interesting finding because some days ago I was poetically thinking about exploiting geographical location and network information/usage patterns for some particular cases that I am investigating now. One of the results that I am particularly aware are related to a time zone aware scheduling approach.

Cheers,
Eli

2006-04-12T21:18:00.000-07:00

IP Design Principles

Recently, I have found a very interesting article that revisited a discussion on the IP packet switching foundations versus circuit switching.

I particularly liked the article structure and agree with some points exposed by the authors. Mainly, the question whether there is an approach to guarantee non-trivial QoS levels over IP other than overprovisioning the links.

Yet, the most important aspect in the article is the attempt to uncurtain new perspectives for packet switching and circuit switching as well.

The paper is short, contains good references and it is definitely worth reading.

Is IP going to take over the world (of communications)?

Regards,
Eli

2006-03-23T19:52:00.000-08:00

Storage Affinity Simulator

Due to several e-mails that I received asking for the simulator that I used to evaluate the Storage Affinity scheduling heuristic, I decided to put the source code available for download here.

Please read the WARNING file inside.

StorageAffinity_Sim.tar.bz2

Enjoy,
./Eli

2006-03-17T20:55:00.000-08:00

Xen and The Art of Hyper-Threading

The quarter is technically over. Here, I decided to post an abstract of some findinds in one of the projects where I had a lot of fun during almost all this time that I have not posted here. :-)

The Performance Impact of Information Gap between Virtual and Physical CPU Capabilities

Theo Hebert, Elizeu Santos-Neto, Andy Seidel

Virtualization is a technique used to create software abstractions of physical hardware architectures to enable more flexible resource utilization and more efficient application execution. Recently, virtualization technologies have regained popularity and created a whole new set of opportunities, that span from software development environments to planetary scale service deployment. However, during the virtualization process, information about architectural components can be lost. This information gap regarding the physical and virtual hardware capabilities may cause performance penalties in the applications executing on top of virtual hardware. In this work we show that the such lack of accuracy exists in the virtual CPU provide by the Xen virtual machine monitor. We focused our investigations on the Hyper-Threading enabled processors. We also describe our solution to alleviates the performance penalty and a discussion about the performance impact of the information gap.

2006-01-26T16:37:00.000-08:00

CPU Inheritance Scheduling

Recently, I remembered a topic that I dedicated a lot of my attention in the end of my undergrad course: CPU Inheritance Scheduling.

The main motivation for Inheritance (Hierarchical or Loadable, if you prefer) Scheduling is the assumption that it is hard to a particular scheduling policy to fulfill the requirements posed by several different target applications.

Therefore, the idea is to allow general purpose systems to easily implement multiple scheduling policies. Furthermore, to have the schedulers organized in a given hierarchy, in the sense that it is possible to reuse the whole scheduling policy logic.

As an example of different application requirements in the same system, we can think of interactive applications have a natural demand for responsiveness (e.g. a text editor, image editing), while batch applications for throughput (e.g. compiling a kernel, running simulations).

Back to the late 90's, the first reference that I have found about it, and that captivated me, was a paper written by Bryan Ford and Sai Susarla (CPU Inheritance Scheduling). In this article, the authors describe the design and implementation of a thread scheduling framework that supports multi-policy scheduling in the FreeBSD system.

Later, an approach that provides yet more flexibility is presented by George Candea Michael B. Jones (Vassal: Loadable Scheduler Support for Multi-Policy Scheduling). In this case, they provide the ability of dynamic loading of scheduling policies. In contrast with the Bryan Ford's paper, the Vassal strategy is better from the point of view that it is not necessary to rely on the scheduling policies made available by the operating system. One could request for loading her own scheduling policy instead. Obviously, this would require the necessary privileges, what turns out in a limited flexibility.

Thus, how about in a system based on virtual machines, where possible harmful user activities will not influence other users? Well, I would primarily think that it might be interesting in a certain degree, however it remains an open question to me.

Maybe, future posts soon. If I have some course projects break. :-)

2006-01-03T17:20:00.000-08:00

In the last Computer Architecture class, it was mentioned that it is easier to find information about the evolution of processors (e.g. number of transistors, performance, etc) than finding the equivalent information about disk and/or memory. This is very interesting because I was wondering about something similar approximately a month ago.

The question in my head was: "Why does not exist a TOP500 ranking of the High Performance Storage infrastructures?".

They have published the www.top500.org highlighting the processing power for years, but few details are included about the storage systems which come together these powerful machines.

Recently, I have found that the IEEE Computer Society Mass Storage Systems Technical Committee is sponsoring an initiative to develop the TOP100io (http://top100io.org/) which is still in the "Call for Contributors" phase.

Particularly, considering that several distributed computational architectures are built today to tackle huge data intensive problems, this ranking might be relevant in guiding future research on high performance storage systems design and performance analysis.

2006-01-02T10:23:00.000-08:00

I used to post some comments about articles that I read in my CiteULike library.

2006-01-02T09:24:00.000-08:00

The purpose of this blog is to be a place where preliminary ideas and opinions about research will be published.

I would appreciate to receive comments about the information posted here.

Cheers,
Eli