scholarly journals Storytelling from an Image Stream Using Scene Graphs

2020 ◽  
Vol 34 (05) ◽  
pp. 9185-9192
Author(s):  
Ruize Wang ◽  
Zhongyu Wei ◽  
Piji Li ◽  
Qi Zhang ◽  
Xuanjing Huang

Visual storytelling aims at generating a story from an image stream. Most existing methods tend to represent images directly with the extracted high-level features, which is not intuitive and difficult to interpret. We argue that translating each image into a graph-based semantic representation, i.e., scene graph, which explicitly encodes the objects and relationships detected within image, would benefit representing and describing images. To this end, we propose a novel graph-based architecture for visual storytelling by modeling the two-level relationships on scene graphs. In particular, on the within-image level, we employ a Graph Convolution Network (GCN) to enrich local fine-grained region representations of objects on scene graphs. To further model the interaction among images, on the cross-images level, a Temporal Convolution Network (TCN) is utilized to refine the region representations along the temporal dimension. Then the relation-aware representations are fed into the Gated Recurrent Unit (GRU) with attention mechanism for story generation. Experiments are conducted on the public visual storytelling dataset. Automatic and human evaluation results indicate that our method achieves state-of-the-art.

Author(s):  
Qiuyuan Huang ◽  
Zhe Gan ◽  
Asli Celikyilmaz ◽  
Dapeng Wu ◽  
Jianfeng Wang ◽  
...  

We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytelling task. Within our framework, the task of generating a story given a sequence of images is divided across a two-level hierarchical decoder. The high-level decoder constructs a plan by generating a semantic concept (i.e., topic) for each image in sequence. The low-level decoder generates a sentence for each image using a semantic compositional network, which effectively grounds the sentence generation conditioned on the topic. The two decoders are jointly trained end-to-end using reinforcement learning. We evaluate our model on the visual storytelling (VIST) dataset. Empirical results from both automatic and human evaluations demonstrate that the proposed hierarchically structured reinforced training achieves significantly better performance compared to a strong flat deep reinforcement learning baseline.


Author(s):  
Bairui Wang ◽  
Lin Ma ◽  
Wei Zhang ◽  
Wenhao Jiang ◽  
Feng Zhang

In this paper, we propose a novel model with a hierarchical photo-scene encoder and a reconstructor for the task of album storytelling. The photo-scene encoder contains two subencoders, namely the photo and scene encoders, which are stacked together and behave hierarchically to fully exploit the structure information of the photos within an album. Specifically, the photo encoder generates semantic representation for each photo while exploiting temporal relationships among them. The scene encoder, relying on the obtained photo representations, is responsible for detecting the scene changes and generating scene representations. Subsequently, the decoder dynamically and attentively summarizes the encoded photo and scene representations to generate a sequence of album representations, based on which a story consisting of multiple coherent sentences is generated. In order to fully extract the useful semantic information from an album, a reconstructor is employed to reproduce the summarized album representations based on the hidden states of the decoder. The proposed model can be trained in an end-to-end manner, which results in an improved performance over the state-of-the-arts on the public visual storytelling (VIST) dataset. Ablation studies further demonstrate the effectiveness of the proposed hierarchical photo-scene encoder and reconstructor.


2020 ◽  
Author(s):  
S. Economides ◽  
C.J. Hourdakis ◽  
C. Pafilis ◽  
G. Simantirakis ◽  
P. Tritakis ◽  
...  

This paper concerns an analysis regarding the performance of X-ray equipment as well as the radiological safety in veterinary facilities. Data were collected from 380 X-ray veterinary facilities countrywide during the on-site regulatory inspections carried out by the Greek Atomic Energy Commission. The analysis of the results shows that the majority of the veterinary radiographic systems perform within the acceptable limits; moreover, the design and shielding of X-ray rooms as well as the applied procedures ensure a high level of radiological safety for the practitioners, operators and the members of the public. An issue that requires specific attention in the optimization process for the proper implementation of veterinary radiology practices in terms of radiological safety is the continuous training of the personnel. The above findings and the regulatory experience gained were valuable decision-making elements regarding the type of the regulatory control of veterinary radiology practices in the new radiation protection framework.


Author(s):  
Sarah Palmeter

In the completion of my practicum at the Public Health Agency of Canada (PHAC) this summer, I worked to develop a surveillance knowledge product to support the national surveillance of developmental disorders. This project used Statistics Canada’s 2017 Canadian Survey on Disability to investigate the burden of developmental disorders in Canada. Developmental disorders are conditions with onset in the developmental period. They are associated with developmental deficits and impairments of personal, social, academic, and occupational function. The project objectives are to estimate the prevalence of developmental disorders in Canadians 15 years of age or older, overall and by age and sex, as well as report on the age of diagnosis, disability severity, and disability co-occurrence in those with developmental disorders. The majority of the analysis has been completed and preliminary results completed, which cannot be released prior to PHAC publication. Although not highly prevalent, developmental disorders are associated with a high level of disability in young Canadians. Early detection and interventions have been shown to improve health and social outcomes among affected individuals. Understanding the burden of developmental disorders in Canada is essential to the development of public health policies and services.


2020 ◽  
Vol 16 (5) ◽  
pp. 860-884
Author(s):  
V.G. Kogdenko ◽  
A.A. Sanzharov

Subject. The article deals with the analysis of suppliers in the public procurement system based on reasonable prequalification parameters. Objectives. The aim is to test the hypothesis about strong reputation characteristics of the winners in the public procurement system and develop a methodology for assessing the reputation of suppliers for prequalification purposes. Methods. We employ general scientific principles and methods of research, like abstraction, generalization of approaches used by domestic and foreign authors for prequalification and assessment of reputation of public procurement participants. Results. To test the hypothesis, we calculated four groups of indicators on corporate, financial, market, and social components of reputation. The methodology was tested on the data obtained from SPARK-Interfax and SPARK-Marketing information resources. Conclusions. The study revealed that not all reputational characteristics of public procurement winners can be regarded as high level. In terms of the corporate component, it is the low level of share capital, indicating the mistrust on the part of owners and their reluctance to invest in the business, and the low percentage of non-current assets. In terms of the market component, it is a low sales growth rate, as well as low return on sales. As to the financial component, it is a low capitalization of winners, low share of long-term debt capital and low credit limit. With respect to the social component, it is a below-average tax burden.


Climate justice requires sharing the burdens and benefits of climate change and its resolution equitably and fairly. It brings together justice between generations and justice within generations. The United Nations Sustainable Development Goals summit in September 2015, and the Conference of Parties to the Framework Convention on Climate Change in Paris in December 2015, brought climate justice center stage in global discussions. In the run up to Paris, Mary Robinson, former president of Ireland and the UN Secretary General’s Special Envoy for Climate Change, instituted the Climate Justice Dialogue. The editors of this volume, an economist and a philosopher, served on the High Level Advisory Committee of the Climate Justice Dialogue. They noted the overlap and mutual enforcement between the economic and philosophical discourses on climate justice. But they also noted the great need for these strands to come together to support the public and policy discourse. This volume is the result.


Author(s):  
Anna Lewandowska ◽  
Grzegorz Rudzki ◽  
Tomasz Lewandowski ◽  
Sławomir Rudzki

(1) Background: As the literature analysis shows, cancer patients experience a variety of different needs. Each patient reacts differently to the hardships of the illness. Assessment of needs allows providing more effective support, relevant to every person’s individual experience, and is necessary for setting priorities for resource allocation, for planning and conducting holistic care, i.e., care designed to improve a patient’s quality of life in a significant way. (2) Patients and Methods: A population survey was conducted between 2018 and 2020. Cancer patients, as well as their caregivers, received an invitation to take part in the research, so their problems and needs could be assessed. (3) Results: The study involved 800 patients, 78% women and 22% men. 66% of the subjects were village residents, while 34%—city residents. The mean age of patients was 62 years, SD = 11.8. The patients received proper treatment within the public healthcare. The surveyed group of caregivers was 88% women and 12% men, 36% village residents and 64% city residents. Subjects were averagely 57 years old, SD 7.8. At the time of diagnosis, the subjects most often felt anxiety, despair, depression, feelings of helplessness (46%, 95% CI: 40–48). During illness and treatment, the subjects most often felt fatigued (79%, 95% CI: 70–80). Analysis of needs showed that 93% (95% CI: 89–97) of patients experienced a certain level of need for help in one or more aspects. (4) Conclusions: Patients diagnosed with cancer have a high level of unmet needs, especially in terms of psychological support and medical information. Their caregivers also experience needs and concerns regarding the disease. Caregivers should be made aware of the health consequences of cancer and consider appropriate supportive care for their loved ones.


2021 ◽  
pp. 145507252199570
Author(s):  
Marjut Salokannel ◽  
Eeva Ollila

Background: Use of snus and snus-like nicotine products is increasing, in particular among young people, in several Nordic countries and Estonia, while snus is legally on the market only in Sweden and Norway. Snus is available in a great variety of tastes and packaging particularly catering for young users. Recently, strong snus-resembling nicotine pouches have emerged on the market. This research investigates the regulatory means to counteract this development. Methods: European Union (EU) and national tobacco control legislation, case law of the European Court of Justice (CJEU) and relevant public health studies are analysed. Results: The research finds that the judgement of the CJEU relating to the sale of snus on Finnish ferries has not been enforced. Permitted large traveller imports for personal use have contributed to wide availability of snus in Finland. Even if the legislation in Sweden is in conformity with the exemption it obtained in the Accession Treaty, the public health impact of snus use for young people in its neighbouring countries has become considerable. Nicotine pouches, -which are not regarded as medical products in terms of medicine legislation, lack harmonised EU-wide regulation. Controlling smuggling across open borders is challenging. Conclusions: The legislation at the EU and national levels should be able to protect young people from new tobacco and nicotine products. It is urgent to harmonise regulation relating to new tobacco and nicotine products taking as a base a high level of protection of health as required in the Treaty on the Functioning of the EU.


Semantic Web ◽  
2020 ◽  
pp. 1-16
Author(s):  
Francesco Beretta

This paper addresses the issue of interoperability of data generated by historical research and heritage institutions in order to make them re-usable for new research agendas according to the FAIR principles. After introducing the symogih.org project’s ontology, it proposes a description of the essential aspects of the process of historical knowledge production. It then develops an epistemological and semantic analysis of conceptual data modelling applied to factual historical information, based on the foundational ontologies Constructive Descriptions and Situations and DOLCE, and discusses the reasons for adopting the CIDOC CRM as a core ontology for the field of historical research, but extending it with some relevant, missing high-level classes. Finally, it shows how collaborative data modelling carried out in the ontology management environment OntoME makes it possible to elaborate a communal fine-grained and adaptive ontology of the domain, provided an active research community engages in this process. With this in mind, the Data for history consortium was founded in 2017 and promotes the adoption of a shared conceptualization in the field of historical research.


Author(s):  
Irfan Uddin

The microthreaded many-core architecture is comprised of multiple clusters of fine-grained multi-threaded cores. The management of concurrency is supported in the instruction set architecture of the cores and the computational work in application is asynchronously delegated to different clusters of cores, where the cluster is allocated dynamically. Computer architects are always interested in analyzing the complex interaction amongst the dynamically allocated resources. Generally a detailed simulation with a cycle-accurate simulation of the execution time is used. However, the cycle-accurate simulator for the microthreaded architecture executes at the rate of 100,000 instructions per second, divided over the number of simulated cores. This means that the evaluation of a complex application executing on a contemporary multi-core machine can be very slow. To perform efficient design space exploration we present a co-simulation environment, where the detailed execution of instructions in the pipeline of microthreaded cores and the interactions amongst the hardware components are abstracted. We present the evaluation of the high-level simulation framework against the cycle-accurate simulation framework. The results show that the high-level simulator is faster and less complicated than the cycle-accurate simulator but with the cost of losing accuracy.


Sign in / Sign up

Export Citation Format

Share Document