Table 2 demonstrates the performance of pre-classified data using the
nearest neighbor method. The highlighted cells represent the TP and
TN prediction with the average accuracy performance at 84.6 percent.
Table 2: Confusion matrix of Nearest Neighbor learner over
We validated different learners for both original data (profile’s content such as age, gender, location) and pre-classified data (extracted
personality factors such as valid, popular, traceable) in order to achieve
more accurate precision. Evaluating both of these sets allows us to
compare the average performance improvement across all three
inputs for the machine learning models.
Our results showed that the overall performance over pre-classified data is higher than using the original data, while incorporating
social network data improves performance yet further. (See Table 3).
Although the diversity of information in pre-classified data is less, it’s
much faster and the prediction performance is more effective than
using the original data by 83.65 to 65.98 percent.
Original data 69.25%
Pre-classified data 86.10%
Learner accuracy 77.68%
Rule Nearest Naïve Overall
Learner Neighbor Bayes Accuracy
66.63% 67.05% 60.99% 65.98%
85.89% 84.59% 78.03% 83.65%
76.26% 75.82% 69.51%
Table 3: Average learner performance comparison when using
original data and pre-classified data.
Real vs. Fake
Our results reveal three important implications. First, they allow us to
clarify our assumption that the levels of honesty and accountability have
a strong correlation when determining real versus fake personas. As
shown in Figure 3, the real nodes are more often associated with both
higher accountability and honesty, while fake users have lower values
for both attributes.
In our identity model (Figure 1), we identified the four possible
types of identity representation: honest and accountable (HA), dishonest and unaccountable (DU), honest and unaccountable (HU), dishonest and accountable (DA). The fraction and frequency of each type
of identity representation are shown in Figure 4.
The results prove our assumptions that:
1. HA highly correlated with real-popular users;
2. DU highly correlated with fake users;
3. HU mainly correlated with real-local users; and
4. DA mainly correlated with real-local and fake users.
Figure 3: Relationship between honesty and accountability
to determine the type of identity.
Figure 4: Four-dimensional identity representation.
Finally, our results show that, despite the important role of honesty in
social interaction, the honesty value (quality in content) has less effect
than accountability value (quantity in interaction) for identity prediction in MySpace.
Friend or Foe?
Our work draws upon research areas in computer sciences, statistics,
sociology, and psychology. Previous work similar to ours has looked at
the validity in self described online identities [ 6, 18]. Some of these
studies showed promise in predicting personality traits with high
accuracy [ 7], but were not able to clarify if the predicted traits are real
The social network analysis we performed was inspired by previous studies on understanding properties of social network structures
[ 3], especially connectivity [ 11], interactions [ 4, 14, 15], and behavior