conclusion

Big data is not just about size. It is also about diversity of data, both in terms of data model (primary key/foreign key versus key/value), as well as consumption pattern (pull versus push), among many other dimensions. This article argues that LINQ is a promising basis for big data. LINQ is both a generalization of relational algebra and has deep roots in category theory—in particular, monads.

With LINQ, queries expressed in
C#, Visual Basic, or JavaScript can be
captured either as code or expression
trees. Either representation can then
be rewritten and optimized and subse-
quently compiled at runtime. We have
also shown how to implement custom
LINQ providers that can run in memory
and over SQL and CoSQL databases,
and we have presented LINQ-friendly
APIs over Web services. It is also pos-
sible to expose streaming data so as to
implement the LINQ standard query
operators, resulting in a single abstrac-
tion that allows developers to query
over all three dimensions of big data.

figure 6. class Queryable implements LinQ standard query operations.

class Queryable<T>
{
Expression This { get; set; }
Queryable(){ This = Expression.Constant(this); }
Queryable<S> Select<S>(Expression<Func<T,S>> f)

acknowledgments

Many thanks to the Cloud Programma-bility team members Savas Parastati-dis, Gert Drapers, Aaron Lahman, Bart de Smet, and Wes Dyer for all the hard work in building the infrastructure and prototypes for all flavors of LINQ and coSQL; to Rene Bouw, Brian Beck-man, and Terry Coatta for helping to improve the readability of this article; and to Dave Campbell and Satya Nadel-la for providing the necessary push to actually write it.

 

figure 7. making the Google chart aPi sequence friendly.

 

string CompileToUri() {

var tt = Title.UrlEncode(); var p = Slices.Unzip(

slices⇒ slices.Select(slice⇒ slice.Legend).Separated-By(“|”), slices⇒ slices.Select(slice⇒ slice.Value).Separated-By(“,”));

Related articles on queue.acm.org

The Pain of Implementing LInQ Providers Oren Eini

http://queue.acm.org/detail.cfm?id=2001564

A Conversation with Erik Meijer and José Blakeley

January 02, 2009

http://queue.acm.org/detail.cfm?id=1394137

A co-Relational Model of Data for Large Shared Data Banks

Erik Meijer, Gavin Bierman

http://queue.acm.org/detail.cfm?id=1961297

 

return string.Format(@“ http://chart.apis.google.com/chart ?cht=p3&chtt={0}&chl=t:{ 1}&chd={ 2}”, tt, p.First, p.Second);

}

here are some convenience constructors for the types Pie and slice, and a getAwaiter method on Pie that triggers the compilation of the sequence to a urI and makes a Web request to google:

class Pie
{
IEnumerable<Slice> Slices; string Title;

class Slice
{
int Value; string Legend;

Now you can create pie charts (and all other google chart types) by writing LINQ queries to generate the data set in a natural way, and then await the image of the chart to come back from the call to http://chart.apis.google.com/chart:

Related Reading

1. c# Query expression translation cheat sheet; http:// bartdesmet.net/blogs/bart/archive/2008/08/30/c-3- 0-query-expression-translation-cheat-sheet.aspx

2. comprehensions with order by and group by; http:// research.microsoft.com/en-us/um/people/simonpj/ papers/list-comp/ index.htm

3. expressions; http://www.cs.cmu.edu/groups/ai/html/ r4rs/ r4rs_6.html#sec28

4. google chart api; http://code.google.com/apis/chart/ image/

5. hadoop; http://hadoop.apache.org/

6. Javascript; https://developer.mozilla.org/en/ Javascript/guide/predefined_core_objects#array_ comprehensions

7. liNQ; http://msdn.microsoft.com/en-us/ netframework/aa904594

8. liNQ to hpc; http://blogs.technet.com/b/ windowshpc/archive/2011/07/07/announcing-linq-to- hpc-beta-2.aspx

9. Monads; http://en.wikipedia.org/wiki/ Monad_%28functional_programming% 29

10. parallel programming with . Net; http://blogs.msdn. com/b/pfxteam/archive/2010/04/04/ 9990343.aspx 11. python; http://www.python.org/dev/peps/pep-0289/ 12. rx (reactive extensions); http://msdn.microsoft.com/ en-us/data/gg577609

13. scala; http://www.scala-lang.org/node/111

14. Xamarin; http://xamarin.com/

var slices = from w in top5
select new Slice( w.Count){ Legend = r.Word };
var image = await new Pie(slices){ Title = “Top 5 words” };

Erik Meijer ( [e-mail]) has been working on “democratizing the cloud” for the past 15 years. he is perhaps best known for his work on the haskell language and his contributions to liNQ and rx (reactive framework).