terms of findings, methodologies, and
context, or Android vs. iOS (see Table 2).
Martin et al.
17 noted that not all stores
provide access to all their reviews, leading to biased findings when studying reviews. To avoid such bias, we collected
all reviews on a daily basis, ensuring we
would include all available reviewers.
However, the Google Play Store provides
access to only the 500 latest reviews for
an app. If more than 500 reviews are received in the 24-hour period between
daily runs of our crawler, then the
crawler does not collect those reviews.
This limitation means we thus offer a
conservative estimate of the number of
reviews for apps that receive more than
500 reviews per 24-hour time period.
We based our Google Play store crawler
on an open source crawler called the
Akdeniz Google Play crawler (https://
github.com/Akdeniz/google-playcrawl-er) to extract app information (such as
app name, user ratings, and reviews).
Running it meant we were simulating
a mobile device over approximately two
months—January 1 to March 2, 2014.
We collected review information
from 12,000 free-to-download apps
from the Google Play store. From among
30 different categories, including photography, sports, and education, we selected the top apps in each category in
the U.S. based on app-analytic company
Distimo’s (acquired by App Annie) ranking of apps for a total of 12,000; Distimo
ranked the top 400 apps for each of the
30 categories. We used Distimo’s Spring
2013 list of top apps. Of the 12,000 top
apps, 1,287 were not accessible during
our two-month crawl because some of
them might have been removed from
the store. We thus collected data from
10,713 top apps, with a total of 11,047
different releases during the studied
Our own selection of top apps might
have biased our results, possibly gener-
alizing to only the top, stable, free apps
in the Google Play store. Nevertheless,
we studied successful apps we felt were
more likely to have a large user base and
receive a large number of reviews, rath-
er than blindly study all apps. We chose
apps that had been popular one year be-
fore we began our study because we were
tencies in apps, analyzing the negative
reviews of apps through topic analy-
sis to identify reasons for users liking
or disliking a given app. Khalid et al.
manually analyzed and categorized one-
and two-star reviews, identifying the is-
sues (such as the hidden cost of using
an app) about which users complained.
Chen et al.
4 proposed the most exten-
sive summarization approach to date,
removing uninformative reviews and
prioritizing the most informative re-
views before presenting a visualization
of the content of reviews. Guzman and
Maalej8 performed natural language
processing techniques to identify app
features in the reviews and leveraged
sentiment analysis to identify whether
users like such features. Our own work
differs from these studies, as it aims to
provide context about when the other
techniques would be needed.
Pagano and Maalej20 and Hoon et
11 analyzed the content of reviews of
both free and paid apps in the Apple
App Store, answering a similar research
question as ours about the number of
received reviews, but there are major
differences between them and us in
Table 1. Our observations on Google Play apps compared to the Pagano and Maalej20 and Hoon et al.
11 observations on the Apple (iOS) App Store.
In the Apple App
Store, from Pagano
In the Apple App Store,
from Hoon et al.
In the Google Play
Store, from us Notes
Reviews received Average of 22 reviews
per day, with 36. 87 for
free apps and 7. 18 for
Median of 50 reviews in
first year for free apps and
30 reviews in first year for
Average of seven reviews
per day, with median of no
reviews per day for free
We found fewer average and median user reviews
compared to Pagano and Maalej21 and more user reviews
than Hoon et al.
11 Reviews were skewed, with median
number of received reviews at 0 and 88% of the studied
apps receiving 20 reviews or fewer per day.
Facebook received 4,275
reviews in one day
(not studied) Only 0.19% of apps
received more than 500
reviews, and the top 100
most-reviewed apps had
6,000 to 43,000 reviews
in the two-month study
Pagano and Maalej21 were the first to observe that some
apps (for them, the Facebook app) might receive a large
number of reviews per day. We were first to explore this
observation—apps receiving a large number of reviews
per day—in depth, finding that while some apps might
receive a large number of reviews, only 0.19% of all
studied apps received more than 500 reviews per day.
Most top apps might not benefit much from automated
approaches that leverage sophisticated techniques (such
as topic modeling) given the small number of reviews
they received and their limited length.
Effect of app
Number of daily reviews
differs by category
Certain categories receive
greater numbers of
reviewers than others
No relation Compared to both iOS studies, we found no relation
between an app’s category and number of received
reviews, once we controlled for number of downloads
and number of releases.
Spike in number
Number of reviews
decreases over time
following a release
(not studied) The standard deviation of
received reviews deviates
from the median directly
following release and
returns back to normal
Both stores showed evidence of spikes in number of
reviews immediately following a new release.
Average length of
Average of 106
characters and median
of 61 characters
Average of 117 characters
and median of 69
Average of 64 characters
and median of 36
Reviews in the Google Play Store were shorter than in the
Apple App Store. Median length of reviews demonstrated
that the distribution of review length is highly skewed,
with long reviews as outliers.