Applications of Web Scraping in Economics and Finance

The 21st-century economy is increasingly built around data. Firms and individuals upload and store enormous amount of data. Most of the produced data is stored on private servers, but a considerable part is made publicly available across the 1.83 billion websites available online. These data can be accessed by researchers using web-scraping techniques. Web scraping refers to the process of collecting data from web pages either manually or using automation tools or specialized software. Web scraping is possible and relatively simple thanks to the regular structure of the code used for websites designed to be displayed in web browsers. Websites built with HTML can be scraped using standard text-mining tools, either scripts in popular (statistical) programming languages such as Python, Stata, R, or stand-alone dedicated web-scraping tools. Some of those tools do not even require any prior programming skills. Since about 2010, with the omnipresence of social and economic activities on the Internet, web scraping has become increasingly more popular among academic researchers. In contrast to proprietary data, which might not be feasible due to substantial costs, web scraping can make interesting data sources accessible to everyone. Thanks to web scraping, the data are now available in real time and with significantly more details than what has been traditionally offered by statistical offices or commercial data vendors. In fact, many statistical offices have started using web-scraped data, for example, for calculating price indices. Data collected through web scraping has been used in numerous economic and finance projects and can easily complement traditional data sources.

On the Self-Shielding in the Unresolved Resonance Range

Resonance behavior is a feature of nuclear reaction cross sections. Resonance density increases with increasing incident particle energy and they begin to overlap, until they can no longer be resolved experimentally, but they still contribute to self-shielding and must be accounted for. This is usually done by representing them with statistical average parameters according to methods and approximations described in standard text-books. Self-shielding factors are commonly used in deterministic transport codes, while statistical Monte Carlo codes use probability tables or multi-band parameters. An exercise was conducted at the International Atomic Energy Agency (IAEA) to validate codes and methods for generating data that account for self-shielding in deterministic and Monte Carlo codes. A simple numerical model problem was defined, considering a sphere of 1 m radius with a 20 MeV isotropic neutron source at the center. The chosen material for testing was 139La from the ENDF/B-VIII.0 library, which clearly showed anomalous behavior.

The life and work of a South African economist: Desmond Hobart Houghton, 1906-76

For forty or so years, from the 1930s to the mid-1970s, Desmond Hobart Houghton was one of South Africa’s most prominent economists, based throughout his academic career at Rhodes University. He belonged to the liberal school of economists who believed in the free market and modernization theory, being particularly influenced by W. Rostow’s stages of growth model which he applied to South Africa. The rural economy, migrant labor and regional development, with a particular focus on the Eastern Cape, were his major research interests. He authored a standard text on the South African economy. This article charts his career and thinking.

Randomised study within a trial (SWAT) to evaluate personalised versus standard text message prompts for increasing trial participant response to postal questionnaires (PROMPTS)

Background Use of a person’s name in a text message has been shown to be effective in instigating behaviour change. We evaluated the effectiveness of a personalised text message (including the recipient’s name) versus a standardised text message for prompting a response from trial participants to complete and return postal follow-up questionnaires. Methods Using a randomised study within a trial (SWAT) embedded within the host GRASP (Getting it Right: Addressing Shoulder Pain) trial, participants who provided a mobile telephone number were randomised (1:1) by a central computer system to receive either (1) a personalised text message which included their name or (2) a standard text message. Text messages were sent by the trial office on the same day as the 6-month GRASP follow-up questionnaire. The primary outcome was questionnaire response rate, defined as the proportion of 6-month GRASP follow-up questionnaires returned by participants. Secondary outcomes included time to response, the proportion of participants sent a reminder follow-up questionnaire, and cost. Results Between March 2017 and May 2019 (recruitment period for GRASP trial), 618 participants were randomised to a personalised (n = 309) or standard (n = 309) text message and all were included in the analysis. The overall questionnaire response rate was 87% (n = 537/618); 90% (n = 277/309) of participants responded in the personalised text message group compared to 84% (n = 260/309) in the standard text message group (relative risk (RR) 1.07; 95% CI 1.00 to 1.13). Participants randomised to receive the personalised text message were more likely to return their initial postal questionnaire than those who received the standard text message (n = 185/309; 60% vs. n = 160/309; 52%) (RR 1.16; 95% CI 1.00 to 1.33); this represents an absolute percentage difference between intervention groups of 8%. Post hoc subgroup analysis showed that males under 65 years were the group most likely to return their initial questionnaire if they received a personalised text message. Conclusion Overall, participants who received a personalised text message were more likely to return their questionnaire than those who received the standard text message. Trial registration GRASP Trial ISRCTN16539266; SWAT Repository ID 35

A new model for automatic text classification

In this paper, a new method for automatic classification of texts is presented. This system includes two phases; text processing and text categorization. In the first phase, various indexing criteria such as bigram, trigram and quad-gram are presented to extract the properties. Then, in the second phase, the W-SMO machine learning algorithm is used to train the system. In order to evaluate and compare the results of the two criteria of accuracy and readability, Macro-F1 and Micro-F1 have been calculated for different indexing methods. The results of experiments performed on 7676 standard text documents of Reuters showed that our proposed method has the best performance compared to the W-j48, Naïve Bayes, K-NN and Decision Tree algorithms.

Does the Acoustic Voice Quality Index (AVQI) Correlate with Perceived Creak and Strain in Normophonic Young Adult Finnish Females?

<b><i>Background:</i></b> The Acoustic Voice Quality Index (AVQI) is a correlate of dysphonia. It has been found to differentiate between dysphonic and normophonic speakers and to indicate the effects of voice therapy. This study investigates how the AVQI reacts towards creak and strain, which are common in normophonic speakers. <b><i>Methods:</i></b> The material was obtained from an earlier study on 104 Finnish female university students (mean age 24.3 years, SD 6.3 years) with no known pathology of voice or hearing and a perceptually normal voice (G = 0 in GRBAS), who were recorded while reading aloud a standard text and sustaining the vowel [a:]. Perceptual analysis for the amount of creak and strain was carried out by 2 expert listeners. In this study, the AVQI v03.01 was analyzed and correlated with perceptual evaluations. Samples with low and high amounts of creak and strain were compared with <i>t</i> tests. <b><i>Results:</i></b> On average, the AVQI was below the threshold value of dysphonia in the Finnish population. The AVQI (ρ = 0.35, <i>p</i> = 0.000) and its subparameters, smoothed cepstral peak prominence (CPPS; ρ = –0.35, <i>p</i> = 0.000) and harmonics-to-noise ratio (HNR; ρ = –0.30, <i>p</i> = 0.002) showed low but significant correlations with creak. Strain had low but significant correlations with spectral Slope (ρ = 0.38, <i>p</i> = 0.000) and Tilt (ρ = –0.40, <i>p</i> = 0.009). The AVQI was lower (better) in samples that were evaluated as having a high amount of strain, but the difference was not significant. Only CPPS differentiated significantly between low and high amounts of creak. <b><i>Conclusion:</i></b> The AVQI does not seem to differentiate between high and low amounts of creak and strain in normophonic speakers.

Randomised study within a trial (SWAT) to evaluate personalised versus standard text message prompts for increasing trial participant response to postal questionnaires (PROMPTS)

Background: Use of a person’s name in a text message has been shown to be effective in instigating behaviour change. We evaluated the effectiveness of a personalised text message (including the recipient’s name), versus a standardised text message for prompting a response from trial participants to complete and return postal follow-up questionnaires.Methods: Using a randomised study within a trial (SWAT) embedded within the host GRASP (Getting it Right: Addressing Shoulder Pain) trial, participants who provided a mobile telephone number were randomised (1:1) by a central computer system to receive either: 1) a personalised text message which included their name; 2) a standard text message. Text messages were sent by the trial office on the same day as the 6-month GRASP follow-up questionnaire. The primary outcome was questionnaire response rate, defined as the proportion of 6-month GRASP follow-up questionnaires returned by participants. Secondary outcomes included time to response, the proportion of participants sent a reminder follow-up questionnaire and cost.Results: Between March 2017 and May 2019 (recruitment period for GRASP trial), 618 participants were randomised to a personalised (n=309) or standard (n=309) text message and all were included in the analysis. The overall questionnaire response rate was 86.9% (n=537/618); 89.6% (n=277/309) of participants responded in the personalised text message group compared to 84.1% (n=260/309) in the standard text message group (Relative Risk (RR) 1.07; 95% CI 1.00 to 1.13). Participants randomised to receive the personalised text message were more likely to return their initial postal questionnaire than those who received the standard text message (n=185/309; 59.9% vs. n=160/309; 51.8%) (RR 1.16; 95% CI 1.00 to 1.33); this represents an absolute percentage difference between intervention groups of 8.1%. Post-hoc subgroup analysis showed that males under 65 years were the group most likely to return their initial questionnaire if they received a personalised text message. Conclusion: Overall, participants who received a personalised text message were more likely to return their questionnaire than those who received the standard text message.Trial Registration: GRASP Trial ISRCTN16539266; SWAT Repository ID 35.

Application of Latent Dirichlet Allocation (LDA) for clustering financial tweets

Sentiment classification is one of the hottest research areas among the Natural Language Processing (NLP) topics. While it aims to detect sentiment polarity and classification of the given opinion, requires a large number of aspect extractions. However, extracting aspect takes human effort and long time. To reduce this, Latent Dirichlet Allocation (LDA) method have come out recently to deal with this issue.In this paper, an efficient preprocessing method for sentiment classification is presented and will be used for analyzing user’s comments on Twitter social network. For this purpose, different text preprocessing techniques have been used on the dataset to achieve an acceptable standard text. Latent Dirichlet Allocation has been applied on the obtained data after this fast and accurate preprocessing phase. The implementation of different sentiment analysis methods and the results of these implementations have been compared and evaluated. The experimental results show that the combined uses of the preprocessing method of this paper and Latent Dirichlet Allocation have an acceptable results compared to other basic methods.

Physics course: tеheory, problems and questions : textbook

The textbook presents theoretical material, problems and questions on the physics course, which will be used during practical classes and for the preparation of control papers, test papers and exam tickets. The presentation of the theory is mainly abstract in nature, so as not to duplicate the content of standard text-books.