two phase
Recently Published Documents


(FIVE YEARS 11568)



2022 ◽  
Vol 431 ◽  
pp. 133939
Langyu Liu ◽  
Shaokun Jiang ◽  
Chunying Zhu ◽  
Youguang Ma ◽  
Taotao Fu

2022 ◽  
Vol 388 ◽  
pp. 111631
Veronica Jauregui Chavez ◽  
Uwe Imke ◽  
Victor Hugo Sanchez-Espinoza

Takashi Hibiki ◽  
Peng Ju ◽  
Somboon Rassame ◽  
Shuichiro Miwa ◽  
Xiuzhong Shen ◽  

2022 ◽  
Vol 40 (4) ◽  
pp. 1-27
Zhongwei Xie ◽  
Ling Liu ◽  
Yanzhao Wu ◽  
Luo Zhong ◽  
Lin Li

This article introduces a two-phase deep feature engineering framework for efficient learning of semantics enhanced joint embedding, which clearly separates the deep feature engineering in data preprocessing from training the text-image joint embedding model. We use the Recipe1M dataset for the technical description and empirical validation. In preprocessing, we perform deep feature engineering by combining deep feature engineering with semantic context features derived from raw text-image input data. We leverage LSTM to identify key terms, deep NLP models from the BERT family, TextRank, or TF-IDF to produce ranking scores for key terms before generating the vector representation for each key term by using Word2vec. We leverage Wide ResNet50 and Word2vec to extract and encode the image category semantics of food images to help semantic alignment of the learned recipe and image embeddings in the joint latent space. In joint embedding learning, we perform deep feature engineering by optimizing the batch-hard triplet loss function with soft-margin and double negative sampling, taking into account also the category-based alignment loss and discriminator-based alignment loss. Extensive experiments demonstrate that our SEJE approach with deep feature engineering significantly outperforms the state-of-the-art approaches.

2022 ◽  
Vol 388 ◽  
pp. 111602
Grant Garrett ◽  
Faith Beck ◽  
Douglas Miller ◽  
Brian Lowery ◽  
Fan-Bill Cheung ◽  

Sign in / Sign up

Export Citation Format

Share Document