Work fragmentation in developer interaction data

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Zou and Godfrey found omnipresence of inactivity in interaction data, interpreted as interruptions, corroborated by programmers.

“One of their findings was the omnipresence of inactivity in the interaction data, that they interpreted as interruptions.They corroborated this with the programmers which agreed that many such periods of inactivity were indeed interruptions.”

• cites [2] 2. L. Zou and M. W. Godfrey, "An industrial case study of program artifacts viewed during maintenance tasks," in Proceedings of WCRE 2006, 2006, pp. 71-82. sourceSupported

May combine two claims.

12% of normal events had a duration >0 seconds and 3% ≥1 hour.

“However, 12% of normal events had a duration > 0 seconds and 3% ≥ 1 hour, these events are outliers.”

Classified as probably not a substantial claim. May combine two claims.

After keeping traces without aggregate information and a little group with aggregate info (max 5 min or two actions), we were left with 6260 traces.

“In order to keep as many traces as possible, we have also kept a little group of traces with aggregate information where the aggregate events have a maximum duration time of five minutes (to minimize uncertainty) or have only two aggregate actions (since the actual event timestamps are known in this case).Thus, we were left with 6260 traces.”

Classified as probably not a substantial claim.

After considering minimum duration 30 minutes, we obtained 4284 useful sessions.

“After this, we considered a minimum duration time of 30 minutes in order to ensure a minimum of activity during a session.In this way, we obtained a final total of 4284 useful sessions to be processed in the next phase.”

Probably not a substantial claim: may not pass S3 (load-bearing).

156 traces (3%) had more than one timezone.

“We found 156 traces (3%) with more than one timezone.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

After preprocessing, edit events increased by 5%, selection events decreased by 20%, total events decreased by 10%.

“edit 452236 475554 +5% selection 658768 527652 -20% total 1111004 1003206 -10%”

Probably not a substantial claim: may not pass S3 (load-bearing).

The total number of selection events was reduced by 45% after joining massive events.

“The total number of selection events was reduced by 45%.”

May combine two claims.

For selections, the effect is moderate for sessions having up to three interruptions, and large to very large for sessions with over four interruptions.

“In selections, the effect is moderate for sessions having up three interruptions, and large to very large for sessions with over four interruptions.”

May combine two claims.

All differences observed between interruption groups are statistically significant with very low p-values according to Mann-Whitney U-test.

“First, all the differences observed are statistically significant with very low p-values (see Table VII) according to the Mann-Whitney U-test.”

May combine two claims.

Prevalence of interruptions by duration: 69% 3-12 min, 18% 12-30 min, 9% 30min-2hr, 4% 2hr-8hr.

“duration ratio examples [3 -12min 69% short pause, answer a question, thinking, looking for interruption [12 -30min 18% coffee break, short meeting, extended interruption [30min -2hr 9% lunch break, a meeting [2hr -8hr 4% extended meeting”

May combine two claims.

In sessions with at least one interruption, code navigation (selections) tends to exceed the frequency of edits, and this is more pronounced when there is at least one prolonged interruption.

“The opposite occurs in sessions with at least one interruption-and is more pronounced when there is at least one prolonged interruption-, where the code navigation tends to exceed the frequency of edits.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Session statistics: total 4284, min 2, 1st Qu 18, median 35, mean 48.91, 3rd Qu 62, max 535.

“total 4,284 min.2 min 1st Qu. 18 min median 35 min mean 48.91 min 3rd Qu. 62 min max.535 min”

May combine two claims.

The relationship between edit ratio and number of interruptions is statistically significant, with effect sizes large to very large.

“A look at the practical and statistical significance of these results (Table VII, bottom) show that the results are (unsurprisingly) statistically significant, and that the observed effect sizes are large to very large.”

May combine two claims.

The effect size of interruption duration on selections per minute is moderate for short interruptions and large for prolonged interruptions.

“In selections, the effect is moderate for short interruptions, and large for prolonged interruptions.”

The quote only partly states this claim.

In sessions with at least one prolonged interruption (≥12 minutes), the number of events per minute is lower than in sessions with only short interruptions (<12 minutes).

“We observe that the number of events per minute is lower in sessions with at least one prolonged interruption.”

May combine two claims.

The edit ratio drops slightly the closer one is to an interruption, and this effect is small but significant.

“The boxplots in Figure 9 show that the edit ratio drops slightly the closer we are to an interruption.The effect is small, but significant (see Table IX).”

May combine two claims.

The local analysis shows three well-defined groups of interruptions, with the two largest having clear effects on activity in the session.

“This local analysis shows that there are indeed three well-defined groups of interruptions, with the two largest of them having clear effects on the activity in the session.”

May combine two claims.

All previous conclusions agree with the UDC data, although the size of the observed effects differ.

“In Section 9, we replicate our first three research questions with the UDC data; we find that all of the previous conclusions agree with this new data, although the size of the observed effects differ.”

May combine two claims.

In sessions with interruptions, the median activity is less than in sessions without interruptions, mainly in edit frequency.

“We observe that the median activity in sessions with interruptions is less than in sessions without interruptions, mainly in the edit frequency.”

May combine two claims.

The mean edit ratio for sessions with short interruptions is 0.4, so the edit ratio close to interruptions is clearly lower in the 5 to 10 minutes around an interruption.

“As a reference, the mean edit ratio for sessions with short interruptions is 0.4, so the edit ratio close to the interruptions are clearly lower in the 5 to 10 minutes around an interruption.”

May combine two claims.

On UDC data, sessions with only short interruptions show a level of edit activity similar to selection activity overall, but with a tendency to rise over time (except a drop in the last segment).

“Sessions with only short interruptions show a level of edit activity similar to selection activity overall, but with a tendency to rise overtime (except a drop in the last segment).”

May combine two claims.

On UDC data, the effect of interruptions on productivity appears less pronounced than in Mylyn; most observed effects are medium, while in Mylyn they were large.

“An important difference is that the effect overall appears to be less pronounced in UDC and we can see it on the effect size tests shown in the Table XVI.Most of the observed effects are medium, while in the Mylyn datasets the effects were large.”

May combine two claims.

On UDC data, the effect sizes for interruption duration are medium to large, less pronounced than in Mylyn.

“The effect size test results in the Table XVII matches the previous observations on a less pronounced effect compared to the Mylyn dataset.The Cohen's d test results tell us that we can label the effect from medium to large and the median values act accordingly to our hypothesis and conclusions with Mylyn.”

May combine two claims.

On UDC data, after a negative interruption, editions per minute is below selections per minute and takes longer to reach average.

“On the contrary to facing a positive interruption, the editions per minute metric is below the selections per minute, and the former takes longer than the latter to reach the average value.”

May combine two claims.

On UDC data, the median editions and selections per minute are greater in sessions without interruptions, and editions are more common than selections.

“The median of editions and selections per minute are greater in sessions without interruptions, and the editions are more common than selections.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

From the query, 4,321,349 unique events were extracted, around 0.18% of the whole dataset.

“From this query we extracted 4,321,349 unique events, which are around 0.18% of the whole dataset.”

May combine two claims.

After splitting into sessions and filtering for at least 30 minutes of productive time, 15,825 useful sessions were obtained from UDC, compared to 4,284 from Mylyn.

“After this, we obtained a total of 28,989 different sessions; but after a filter to keep only those with at least 30 minutes of productive time (as we did with the Mylyn dataset), we ended up with 15,825 useful sessions, in contrast with the 4,284 we had from Mylyn.”

May combine two claims.

On UDC data, close to the interruption the frequency of editions and selections drastically drop, and in a radius of around 10 minutes both metrics are below the overall median.

“We observed that close to the interruption the frequency of editions and selections drastically drop, and in a radius of around 10 minutes both metrics are below the overall median.”

May combine two claims.

On average, the peak lasts for 12 minutes, while the maximum registered is of 145 minutes.

“On average, the peak lasts for 12 minutes, while the maximum registered is of 145 minutes.”

May combine two claims.

On UDC data, when facing zero interruptions, editions and selections per minute are greater than with one or more interruptions, and as more interruptions occur these metrics gradually decrease.

“When facing zero interruptions the editions and selections per minute are greater than with one or more interruptions, and as more interruptions occur this metrics gradually decrease.”

May combine two claims. The quote only partly states this claim.

The proportion of events after classification is: edition 34.6%, selection 31.1%, system 31.5%, control 0.8%.

“The proportion of events after the classification is shown in the Table XIII.We used an iterative process involving two of the authors of the paper to arrive to this result; the process was iterative, until we reached consensus.”

May combine two claims.

On UDC data, both positive and negative interruptions have a proportion of 44% each, and neutrals 12%.

“Both positive and negative interruptions have a proportion of 44% each of all the interruptions, and the neutrals stand with the 12%, and they are similar to the proportions on Mylyn.”

May combine two claims.

On UDC data, editions per minute recover quickly, approximately 5 minutes after the interruption, but selections per minute take longer to reach the global average and tend to be below average for an extended period.

“The editions per minute recover quickly, approximately 5 minutes after the interruption, but the selections per minute take a little longer to reach the global average and this metric tend to be below the average for an extended period.”

May combine two claims.

On UDC data, when facing at least one prolonged interruption, the metrics drop compared to short interruptions and selections are more frequent than edit events.

“When facing at least one prolonged interruption we can see the metrics drop in comparison with short interruptions and the selections are now more frequent than the edit events.”

May combine two claims.

The results indicate a clear diminish of productivity when facing interruptions, but with a high number of interruptions the change of productivity stops decreasing.

“Furthermore, the results indicate a clear diminish of productivity when facing interruptions, but with a high number of interruptions the change of productivity stops decreasing.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The latter actually occurs in the results but the effect is too subtle.

“The latter actually occurs in the results but the effect is too subtle.”

May combine two claims.

This analysis also found two well-defined patterns around interruptions: interruptions yielding negative effects (consistent with a real-life interruption involving an expensive context switch), and interruptions with positive effects (consistent with information-seeking behavior).

“This analysis also found two well-defined patterns around interruptions: interruptions yielding a negative effects (consistent with a real-life interruption involving an expensive context switch), and interruptions with positive effects (consistent with information-seeking behavior).”

May combine two claims.

The additional evidence uncovered in the UDC supports this hypothesis, since different activities appear to take place in both types of interruptions.

“the additional evidence uncovered in the UDC does support this hypothesis, since different activities appear to take place in both types of interruptions.”

Classified as probably not a substantial claim. May combine two claims.

Our local analysis uncovered two well-defined patterns around gaps of inactivity: ones hypothesized as actual interruptions (negative effect post-interruptions), and ones hypothesized as more likely information-seeking activities (positive effect post-interruption).

“We note that our local analysis uncovered two well-defined patterns around gaps of inactivity: ones that we hypothesize are actual interruptions (negative effect post-interruptions), and the ones that we hypothesize are more likely information-seeking activities (positive effect postinterruption).”

May combine two claims.

Our additional dataset (the UDC) comes from a much larger and more varied sample of users, and does show similar behavior, which raises confidence in our results.

“Our additional dataset (the UDC) comes from a much larger and more varied sample of users, and does show similar behavior, which raises confidence in our results.”

May combine two claims.

A subsequent local analysis around interruptions confirmed these results as the activity around interruptions was found to be lower than average.

“A subsequent local analysis around interruptions confirmed these results as the activity around interruptions was found to be lower than average.”

May combine two claims.

We found a higher productivity when the interruption is positive and identified a positive evolution of the metrics as the programmer advances through the Recovery Time, reaching a peak of productivity between the last minutes of the Recovery Time and 10 minutes before the next interruption happens.

“We first analyzed the Recovery Time in detail, finding a higher productivity when the interruption is positive and we identified a positive evolution of the metrics as the programmer advances through this stage, reaching a peak of productivity between the last minutes of the Recovery Time and 10 minutes before the next interruption happens.”

May combine two claims.

After a negative interruption the most common events are debugging, searching and navigation around classes, which tell us about a doubtful and uncertain behavior from the programmer.

“In contrast, after a negative interruption the most common events are debugging, searching and navigation around classes, which tell us about a doubtful and uncertain behavior from the programmer.”

Classified as probably not a substantial claim. May combine two claims.

The partition of data into sessions does not alter the results because we are interested in values by minute.

“Additionally, the partition of the data into sessions do not alter the results since we are interested in the values by minute, so having longer sessions do not necessarily increase the number of events per minute.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Trading quality for quantity allows exploring different aspects and generating hypotheses for subsequent studies.

“Trading quality for quantity allows us to explore different aspects of the problem, and to generate hypotheses for subsequent studies of work fragmentation and interruptions in software development.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Information workers average 3 minutes on a task before switching to another task.

“An observational study by Gonzalez and Mark found that information workers experience high level of multitasking, averaging 3 minutes on a task before switching to another task [3].”

• cites [3] 3. V. M. González and G. Mark, ""constant, constant, multi-tasking craziness": managing multiple working spheres," in Proceedings of CHI 2004, 2004, pp. 113-120. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study of Parnin and Rugaber is closest in amount of data, but its large-dataset analysis focused principally on edit lag, while this paper investigates productivity indicators along entire session.

“The study of Parnin and Rugaber is the closest to ours in terms of amount of data considered.However, the analysis performed on large datasets in their study focused principally on the edit lag metric, while we investigate metrics which are indicators of productivity along the entire session.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A worker needs on average 25 minutes to get back on an interrupted task.

“a study of 24 information workers found that a worker needs on average 25 minutes to get back on an interrupted task [5].”

• cites [5] 5. G. Mark, V. M. González, and J. Harris, "No task left behind?: examining the nature of fragmented work," in Proceedings of CHI 2005, 2005, pp. 321-330. sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Mylyn task contexts are a rather reliable account of IDE usage over time.

“The task context contains the entire interaction history since a developer activated the task he or she was working on, and as such is a rather reliable account of IDE usage over time (barring a few issues explained below).”

May combine two claims.

Aggregate events in Mylyn data lose specific time besides range, because Mylyn does not register all user events for scalability.

“Whenever an aggregation occurs, the event is expanded to include two timestamps defining a range of time, instead of a single timestamp, and a number of events.Accordingly, these aggregate events lose their specific time besides the range of time.The reason of this is that for scalability-in terms of storage-Mylyn does not register all the user events.”

Classified as probably not a substantial claim.

Short interruptions typically last around 3 minutes.

“This is based on previous work where we observed that short interruptions lasted usually this long [3].”

• cites [3] 3. V. M. González and G. Mark, ""constant, constant, multi-tasking craziness": managing multiple working spheres," in Proceedings of CHI 2004, 2004, pp. 113-120. sourceNot checked — no full text of the source

May combine two claims.

Very long events have duration much larger than mean, mainly in aggregate events, due to Mylyn registering end time when task resumed after long downtime.

“These events have a duration time (|EndT ime -StartT ime|) much larger than the mean.This mainly occurs in aggregate events.We believe that this issue is due to factors related to Mylyn, since it can register the end time of an event when the task is resumed after a long downtime.”

Probably not a substantial claim: may not pass S4 (based).

Most prior studies operate on small datasets.

“As above, most of these studies operate on small datasets.”

May combine two claims.

The duration of very long events is overstated; finding their actual duration is not trivial.

“Consequently, the duration of these events is overstated; however, finding their actual duration is not trivial.”

May combine two claims.

The literature lacks studies targeting software developers and large-scale data.

“If the literature contains extensive studies of interaction workers, it lacks in two aspects: (1) most studies target information workers (fewer target the specific population of software developers), and (2) the studies are usually field studies, which are limited in the amount of data they contain.”

May combine two claims.

Parnin and Rugaber evidenced edit lag in majority of large sample of development sessions, showing developers need time to resume work after interruption.

“Parning and Rugaber evidenced the presence of an edit lag in the majority of a large sample of development session spanning several datasets, hence showing that developers need time to resume work after an interruption [9].”

• cites [9] 9. C. Parnin and S. Rugaber, "Resumption strategies for interrupted programming tasks," Software Quality Journal, vol. 19, no. 1, pp. 5-34, 2011.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

Work fragmentation impact may be worse for software developers because they build complex mental models that are more sensitive to interruptions and costly to rebuild.

“Work fragmentation is indeed an important phenomenon within the context of modern software development, and the impact there may be even worse since developers build and maintain complex mental models of the software they are working on-these models may be more sensible to interruptions, and costly to rebuild.”

May combine two claims.

The observation that time series without interruptions have more edits in the middle matches the hypothesis that developers need time to build their mental model and are less productive early in a session.

“This observation matches the hypothesis that developers need time to build their mental model for their task, and are less productive early in a session.”

May combine two claims.

Activity Theory models consider interruptions after ~3 minutes of sustained attention at action level and after ~12 minutes at activity level.

“This study presented work fragmentation at two different levels: actions and activities.Interruptions originated after a period of around three minutes of sustained attention to the previous action were considered when people were switching at the level of interactions with artifacts of people.Interruptions originated after a period of twelve minutes of sustained attention to a previous activity w”

• cites [33] 33. R. Robbes and M. Lanza, "Characterizing and understanding development sessions," in Proceedings of ICPC 2007, 2007, pp. 155-166. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The finding that the inverse relationship between productivity and interruption duration is more pronounced with longer interruptions agrees with the literature for information workers.

“This agrees with the literature for information workers.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The drop in activity before an interruption may be because the programmer found a problem and engaged in debugging or asking teammates, which reduces IDE activity.

“This is might be because of two reasons: the programmer could have found a problem while coding, who at first will try to solve it by reading the code, switching out from the IDE, navigating the call stack and debugging, before going to ask another teammate, according to LaToza et al. [35]; this set of actions end up with an interruption and reduce the observed activity within the IDE.”

• cites [35] 35. J. Cohen, "The earth is round (p < 0.5)," American Psychologist, vol. 49, no. 12, pp. 997-1003, 1994. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

The increasing trend of activity after an interruption reflects the programmer immersing again into the programming task.

“We hypothesize that the programmer is immersing again into the programming task, increasing progressively the activity as represented by the number of edits and selections.”

May combine two claims.

These results support previous observations at a finer level and corroborate the literature saying that time is needed to start or get back on task.

“These results support our previous observations at a finer level, and corroborate the literature saying that time is needed to start or get back on task.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The finding that interruptions inversely relate to activity and productivity agrees with previous literature on the harmfulness of multitasking, work fragmentation, and interruptions.

“This finding agrees with the previous literature on the harmfulness of multitasking, work fragmentation, and interruptions.”

May combine two claims.

53% of the interruptions had 30 minutes before and after the interruption and were selected for the analysis; the remaining 47% could not be used due to lack of time around them.

“In total, 53% of the interruptions had 30 minutes before and after the interruption and were selected for the analysis.The rest 47% of the interruptions can not be used on this analysis due to the lack of time around them.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The drop in edit ratio near interruptions matches Parnin's observations with the edit lag.

“This matches the observations Parnin made with the edit lag [9].”

• cites [9] 9. C. Parnin and S. Rugaber, "Resumption strategies for interrupted programming tasks," Software Quality Journal, vol. 19, no. 1, pp. 5-34, 2011.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

In sessions with interruptions, the edit lag behavior is not visible; each session may have several edit lags after each interruption, equally distributed, resulting in flat lines.

“This behavior is not visible in sessions with interruptions; indeed, each session may have several edit lags (after each interruptions), and those would be equally distributed over the entire duration of the session, resulting in the”

May combine two claims.

Interruptions and user activity follow inverse relationships, and they are quite pronounced.

“This reinforces our impressions that interruptions and user activity follow inverse relationships, and that they are quite pronounced.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

To have the same amount of information we have with UDC it would require to monitor the user activity for a year or two and have a record of all the activities performed, like the number and duration of the interruptions.

“For example, to have the same amount of information we have with UDC it would require to monitor the user activity for a year or two and have a record of all the activities performed, like the number and duration of the interruptions.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Analyzing interaction data is a convenient way to understand almost the same kind of events on the environment.

“On the other hand, analyzing interaction data is a convenient way to understand almost the same kind of events on the environment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The small size of the Mylyn dataset may have favored outliers, affecting the results.

“The (comparatively) small size of the Mylyn dataset may have favored outliers: since there is data from dozens of developers only in the Mylyn dataset, a few outliers may have affected the results.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

These findings helped us understand the response to work fragmentation according to the recorded user activity, and it was possible to make a relation between the user's data and the phenomena that occur on the working places described in the studies.

“These findings helped us understand the response to work fragmentation according to the recorded user activity, and it was possible to made a relation between the user's data and the phenomena that occur on the working places describe on the studies.”

May combine two claims.

Editions per minute describes the effect better and selections do not show a major change.

“The editions per minute describes better the effect and the selections do not show a major change.”

Probably not a substantial claim: may not pass S3 (load-bearing).

As the number of switches between tasks increases, the effectiveness degrades in a linear fashion.

“Moreover, as the number of switches between tasks increases, the effectiveness degrades in a linear fashion [42].”

• cites [42] 42. S. Aral, E. Brynjolfsson, and M. Van Alstyne, "Information, technology, and information worker productivity," Information Systems Research, vol. 23, no. 3-part-2, pp. 849-867, 2012. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The pre-processed UDC data by Murphy-Hill is cleaned and preprocessed, making transformation simple.

“We used the pre-processed data by Murphy-Hill, that is published on Google BigQuery [39].This is an alternative version of the original UDC dataset, that is cleaned and preprocessed, so that the transformation phase is simple and focused on our needs.”

• cites [39] 39. E. J. Keogh, "Exact indexing of dynamic time warping," Knowledge and Information Systems, vol. 7, pp. 358-386, 2005. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

This could be reflecting what observational studies tell us about multitasking, which is that with higher number of interruptions (increased multitasking) the productivity stops being affected and might increase.

“This could be reflecting what observational studies tell us about multitasking, which is that with higher number of interruptions (increased multitasking) the productivity stops being affected and might increase.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Usage data describes the history of activities of the programmer and it is possible to analyze the effect of events, as shown in this work.

“Usage data describes the history of activities of the programmer and it is possible to analyze the effect of events, as shown in this work.”

Classified as probably not a substantial claim. May combine two claims.

The problem with analyzing interaction data is the lack of context of the events and, in our case, the lack of information of the programmers to enrich the results.

“The problem with this is the lack of context of the events and, in our case, the lack of information of the programmers to enrich the results.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The classification of events into edition and selection was done through an iterative process involving two authors until consensus.

“We used an iterative process involving two of the authors of the paper to arrive to this result; the process was iterative, until we reached consensus.”

May combine two claims.

The models are not directly comparable: Mylyn records edition and selection events directly, while the UDC records a broader array of user activity; classification of UDC events does not exactly replicate the Mylyn model.

“The models are not directly comparable: Mylyn records edition and selection events directly, while the UDC records a broader array of user activity.”

May combine two claims.

At some level, switching between tasks can provide the needed arousal and alertness to improve productivity, but higher levels of multitasking can be detrimental and cause a negative effect.

“At some level, switching between tasks can provide the needed arousal and alertness to improve the productivity, but higher levels of multitasking can be detrimental and cause a negative effect.”

• cites [42] 42. S. Aral, E. Brynjolfsson, and M. Van Alstyne, "Information, technology, and information worker productivity," Information Systems Research, vol. 23, no. 3-part-2, pp. 849-867, 2012. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All the observations go in the same direction, which lends more confidence in the conclusions.

“The fact that all the observations go in the same direction lends us more confidence in our conclusions.”

Classified as probably not a substantial claim. May combine two claims.

We cannot guarantee that an interruption was an actual interruption of work; it could also be a change to a Web design tool, server configuration tool or the browser, which are common activities in Web developers and not necessarily detrimental to productivity.

“Hence, we can not guarantee that an interruption was an actual interruption of work; due to the definition of an interruption, it could also be a change to a Web design tool, server configuration tool or the browser, which are common activities in Web developers and not necessarily detrimental to productivity.”

Classified as probably not a substantial claim.

The lower edit ratio after negative interruptions is in accordance with the hypothesis that the programmer may be rebuilding context after an unwanted task switch.

“negative interruptions have a lower edit ratio after the interruption (in accordance with the hypothesis that the programmer may be rebuilding his context after an unwanted task switch)”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Implementing an observational study of this kind can be very expensive, for the time and material need.

“However, implementing a study of this kind can be very expensive, for the time and material need.”

Classified as probably not a substantial claim.

After selecting interruptions with at least 15 minutes before another interruption, 5,684 valid interruptions were obtained from a total of 110,077.

“It was required to make a selection of those interruptions which had a period afterwards of at least 15 minutes before another interruption.After this query we obtained 5,684 valid interruptions from a total of 110,077.”

May combine two claims.

The first five minutes after an interruption and the moment before the next one are the phases where productivity is more affected, while the phases around the peak are where productivity is more favorable.

“From this we can conclude that the first five minutes after an interruption and the moment before the next one are the are the phases where productivity is more affected, while the phases around the peak are where the productivity is more favorable.”

May combine two claims.

These categories of events agree with the previous hypothesis that positive interruptions are related to information seeking, while negative interruptions are related to actual task switches where context needs to be rebuilt afterwards.

“These categories of events agree with our previous hypothesis that positive interruptions are related to information seeking [9,35], while negative interruptions are related to actual task switches where context needs to be rebuilt afterwards.”

• cites [9] 9. C. Parnin and S. Rugaber, "Resumption strategies for interrupted programming tasks," Software Quality Journal, vol. 19, no. 1, pp. 5-34, 2011.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [35] 35. J. Cohen, "The earth is round (p < 0.5)," American Psychologist, vol. 49, no. 12, pp. 997-1003, 1994. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The duration of the peak varies among working spheres.

“The duration of the peak varies among the also called working spheres, as mentioned by Gonzalez and Mark [5].”

• cites Gonzalez and Mark [5]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The peak phase is where productivity should be reaching its maximum, as programmers are not taking actions to retake the lost mental model or preparing for an interruption.

“The peak phase is where the productivity should be reaching its maximum, as the programmers are not taking actions to retake the lost mental model or preparing for an interruption.”

Classified as probably not a substantial claim.

The recovery time is established as the 15-minute period after an interruption, without considering the time consumed by the interruption itself.

“We established the recovery time as the 15 minute period after an interruption, without considering the time consumed by the interruption itself.The length of the period was chosen according to the observations by Solingen et al. [40].”

• cites [40] 40. W. Snipes, E. Murphy-Hill, T. Fritz, M. Vakilian, K. Damevski, A. R. Nair, and D. Shepherd, Analyzing Software Data. Morgan Kaufmann, 2015, ch. A Practical Guide to Analyzing IDE Usage Data. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Other effect size measures such as the common language effect size or A12 may slightly alter the discussion.

“Other effect size measures such as the common language effect size [44] or A 12 [45] may slightly alter the discussion.”

• cites [44] 44. R. Robbes and M. Lanza, "Improving code completion with program history," Autom. Softw. Eng., vol. 17, no. 2, pp. 181-212, 2010. sourceNot checked — no full text of the source
• cites [45] 45. K. O. McGraw and S. P. Wong, "A Common Language Effect Size Statistic," Psychological Bulletin, vol. 111, pp. 361-365, 1992. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

These findings agree with the literature on information workers and software developers.

“These findings agree with the literature on information workers and software developers.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

We were careful not to conclude that every gap of activity is an actual interruption.

“As stated in the introduction, we were careful not to conclude that every gap of activity is an actual interruption.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Our separation of sessions in short and long interruptions is not perfect, as it is based on the presence of at least one long interruption, and nothing more.

“We acknowledge that our separation of sessions in short and long interruptions is not perfect, as it is based on the presence of at least one long interruption, and nothing more.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We used Cohen's d, a parametric effect size measure, as it has defined thresholds allowing easier interpretation of the results.

“We used Cohen's d, a parametric effect size measure, as it has defined thresholds allowing easier interpretation of the results.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Other indicators of productivity (such as actual changes) may yield different results.

“However we can not exclude that other indicators of productivity (such as actual changes [43]) may yield different results.”

• cites [43] 43. R. F. Adler and R. Benbunan-Fich, "Juggling on a high wire: Multitasking effects on performance," International Journal of Human-Computer Studies, vol. 70, no. 2, pp. 156-168, 2012. sourceNot checked — no full text of the source

May combine two claims.

The gathered data corresponds to a limited group of programmers, which use both Mylyn and Eclipse regularly.

“The gathered data corresponds to a limited group of programmers, which use both Mylyn and Eclipse regularly.”

Classified as probably not a substantial claim. May combine two claims.

We decided not to count workbench activation and deactivation events as 'system' events, and thus only use them to define an interruption or the end of a session.

“Therefore, we decided not count them as”

May combine two claims.

These findings support our hypotheses that positive interruptions are due to information-seeking, while negative interruptions are due to context rebuilding after an actual interruption.

“These findings support out hypotheses that positive interruptions are due to information-seeking, while negative interruptions are due to context rebuilding after an actual interruption.”

May combine two claims.

We have not found evidence of bias due to the exclusion of sessions with aggregated information, but it may still be present.

“We have not found evidence of bias due to this but it may still be present.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Other factors present in the group of sessions with at least one long interruption may contribute to the observed effect.

“Other factors present in the group of sessions with at least one long interruption may contribute to the observed effect (for instance, these sessions may also have more interruptions overall).”

Probably not a substantial claim: may not pass S3 (load-bearing).

A large variation of the thresholds (8 hours of inactivity, 30 minutes minimum session, 3 minutes minimum interruption, ≥12 minutes prolonged interruptions) might significantly alter the results.

“However, a large variation of these thresholds might significantly alter the results.”

Classified as probably not a substantial claim.

Our classification of events is a possible threat, as it was based on our own criteria of what should be an Edition and Selection event.

“Additionally, our classification of events is a possible threat, as it was based on our own criteria of what should be an Edition and Selection event.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The same threats about the metrics, implications and statistical conclusions apply on the results observed with the UDC.

“The same threats about the metrics, implications and statistical conclusions apply on the results observed with the UDC.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We took special care when establishing the classification, using an iterative process involving two authors of this paper.

“We took special care when establishing the classification, using an iterative process involving two authors of this paper.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We ran tests of what kind of data was recorded for specific usage scenarios of the IDE in order to better understand the nature of the dataset when particular events were unclear.

“We also ran tests of what kind of data was recorded for specific usage scenarios of the IDE in order to better understand the nature of the dataset when particular events were unclear.”