Hugging Face's logo

zeroMN
/
SHMT

Text Generation
Transformers
English
Chinese
transformer
multimodal
vqa
text
audio
Eval Results
Inference Endpoints
Model card Files Community
1
SHMT / README.md
zeroMN's picture
zeroMN
Update README.md
367ab04 verified
preview code
|
raw
history blame contribute delete
10.9 kB
metadata
datasets:
  - nyu-mll/glue
  - aps/super_glue
  - facebook/anli
  - tasksource/babi_nli
  - zeroMN/AVEdate
  - sick
  - snli
  - scitail
  - hans
  - alisawuffles/WANLI
  - tasksource/recast
  - sileod/probability_words_nli
  - joey234/nan-nli
  - pietrolesci/nli_fever
  - pietrolesci/breaking_nli
  - pietrolesci/conj_nli
  - pietrolesci/fracas
  - pietrolesci/dialogue_nli
  - pietrolesci/mpe
  - pietrolesci/dnc
  - pietrolesci/recast_white
  - pietrolesci/joci
  - pietrolesci/robust_nli
  - pietrolesci/robust_nli_is_sd
  - pietrolesci/robust_nli_li_ts
  - pietrolesci/gen_debiased_nli
  - pietrolesci/add_one_rte
  - tasksource/imppres
  - hlgd
  - paws
  - medical_questions_pairs
  - Anthropic/model-written-evals
  - truthful_qa
  - nightingal3/fig-qa
  - tasksource/bigbench
  - blimp
  - cos_e
  - cosmos_qa
  - dream
  - openbookqa
  - qasc
  - quartz
  - quail
  - head_qa
  - sciq
  - social_i_qa
  - wiki_hop
  - wiqa
  - piqa
  - hellaswag
  - pkavumba/balanced-copa
  - 12ml/e-CARE
  - art
  - winogrande
  - codah
  - ai2_arc
  - definite_pronoun_resolution
  - swag
  - math_qa
  - metaeval/utilitarianism
  - mteb/amazon_counterfactual
  - SetFit/insincere-questions
  - SetFit/toxic_conversations
  - turingbench/TuringBench
  - trec
  - tals/vitaminc
  - hope_edi
  - strombergnlp/rumoureval_2019
  - ethos
  - tweet_eval
  - discovery
  - pragmeval
  - silicone
  - lex_glue
  - papluca/language-identification
  - imdb
  - rotten_tomatoes
  - ag_news
  - yelp_review_full
  - financial_phrasebank
  - poem_sentiment
  - dbpedia_14
  - amazon_polarity
  - app_reviews
  - hate_speech18
  - sms_spam
  - humicroedit
  - snips_built_in_intents
  - hate_speech_offensive
  - yahoo_answers_topics
  - pacovaldez/stackoverflow-questions
  - zapsdcn/hyperpartisan_news
  - zapsdcn/sciie
  - zapsdcn/citation_intent
  - go_emotions
  - allenai/scicite
  - liar
  - relbert/lexical_relation_classification
  - tasksource/linguisticprobing
  - tasksource/crowdflower
  - metaeval/ethics
  - emo
  - google_wellformed_query
  - tweets_hate_speech_detection
  - has_part
  - blog_authorship_corpus
  - launch/open_question_type
  - health_fact
  - commonsense_qa
  - mc_taco
  - ade_corpus_v2
  - prajjwal1/discosense
  - circa
  - PiC/phrase_similarity
  - copenlu/scientific-exaggeration-detection
  - quarel
  - mwong/fever-evidence-related
  - numer_sense
  - dynabench/dynasent
  - raquiba/Sarcasm_News_Headline
  - sem_eval_2010_task_8
  - demo-org/auditor_review
  - medmcqa
  - RuyuanWan/Dynasent_Disagreement
  - RuyuanWan/Politeness_Disagreement
  - RuyuanWan/SBIC_Disagreement
  - RuyuanWan/SChem_Disagreement
  - RuyuanWan/Dilemmas_Disagreement
  - lucasmccabe/logiqa
  - wiki_qa
  - tasksource/cycic_classification
  - tasksource/cycic_multiplechoice
  - tasksource/sts-companion
  - tasksource/commonsense_qa_2.0
  - tasksource/lingnli
  - tasksource/monotonicity-entailment
  - tasksource/arct
  - tasksource/scinli
  - tasksource/naturallogic
  - onestop_qa
  - demelin/moral_stories
  - corypaik/prost
  - aps/dynahate
  - metaeval/syntactic-augmentation-nli
  - tasksource/autotnli
  - lasha-nlp/CONDAQA
  - openai/webgpt_comparisons
  - Dahoas/synthetic-instruct-gptj-pairwise
  - metaeval/scruples
  - metaeval/wouldyourather
  - metaeval/defeasible-nli
  - tasksource/help-nli
  - metaeval/nli-veridicality-transitivity
  - tasksource/lonli
  - tasksource/dadc-limit-nli
  - ColumbiaNLP/FLUTE
  - tasksource/strategy-qa
  - openai/summarize_from_feedback
  - tasksource/folio
  - yale-nlp/FOLIO
  - tasksource/tomi-nli
  - tasksource/avicenna
  - stanfordnlp/SHP
  - GBaker/MedQA-USMLE-4-options-hf
  - sileod/wikimedqa
  - declare-lab/cicero
  - amydeng2000/CREAK
  - tasksource/mutual
  - inverse-scaling/NeQA
  - inverse-scaling/quote-repetition
  - inverse-scaling/redefine-math
  - tasksource/puzzte
  - tasksource/implicatures
  - race
  - tasksource/race-c
  - tasksource/spartqa-yn
  - tasksource/spartqa-mchoice
  - tasksource/temporal-nli
  - riddle_sense
  - tasksource/clcd-english
  - maximedb/twentyquestions
  - metaeval/reclor
  - tasksource/counterfactually-augmented-imdb
  - tasksource/counterfactually-augmented-snli
  - metaeval/cnli
  - tasksource/boolq-natural-perturbations
  - metaeval/acceptability-prediction
  - metaeval/equate
  - tasksource/ScienceQA_text_only
  - Jiangjie/ekar_english
  - tasksource/implicit-hate-stg1
  - metaeval/chaos-mnli-ambiguity
  - IlyaGusev/headline_cause
  - tasksource/logiqa-2.0-nli
  - tasksource/oasst2_dense_flat
  - sileod/mindgames
  - metaeval/ambient
  - metaeval/path-naturalness-prediction
  - civil_comments
  - AndyChiang/cloth
  - AndyChiang/dgen
  - tasksource/I2D2
  - webis/args_me
  - webis/Touche23-ValueEval
  - tasksource/starcon
  - PolyAI/banking77
  - tasksource/ConTRoL-nli
  - tasksource/tracie
  - tasksource/sherliic
  - tasksource/sen-making
  - tasksource/winowhy
  - tasksource/robustLR
  - CLUTRR/v1
  - tasksource/logical-fallacy
  - tasksource/parade
  - tasksource/cladder
  - tasksource/subjectivity
  - tasksource/MOH
  - tasksource/VUAC
  - tasksource/TroFi
  - sharc_modified
  - tasksource/conceptrules_v2
  - metaeval/disrpt
  - tasksource/zero-shot-label-nli
  - tasksource/com2sense
  - tasksource/scone
  - tasksource/winodict
  - tasksource/fool-me-twice
  - tasksource/monli
  - tasksource/corr2cause
  - lighteval/lsat_qa
  - tasksource/apt
  - zeroshot/twitter-financial-news-sentiment
  - tasksource/icl-symbol-tuning-instruct
  - tasksource/SpaceNLI
  - sihaochen/propsegment
  - HannahRoseKirk/HatemojiBuild
  - tasksource/regset
  - tasksource/esci
  - lmsys/chatbot_arena_conversations
  - neurae/dnd_style_intents
  - hitachi-nlp/FLD.v2
  - tasksource/SDOH-NLI
  - allenai/scifact_entailment
  - tasksource/feasibilityQA
  - tasksource/simple_pair
  - tasksource/AdjectiveScaleProbe-nli
  - tasksource/resnli
  - tasksource/SpaRTUN
  - tasksource/ReSQ
  - tasksource/semantic_fragments_nli
  - MoritzLaurer/dataset_train_nli
  - tasksource/stepgame
  - tasksource/nlgraph
  - tasksource/oasst2_pairwise_rlhf_reward
  - tasksource/hh-rlhf
  - tasksource/ruletaker
  - qbao775/PARARULE-Plus
  - tasksource/proofwriter
  - tasksource/logical-entailment
  - tasksource/nope
  - tasksource/LogicNLI
  - kiddothe2b/contract-nli
  - AshtonIsNotHere/nli4ct_semeval2024
  - tasksource/lsat-ar
  - tasksource/lsat-rc
  - AshtonIsNotHere/biosift-nli
  - tasksource/brainteasers
  - Anthropic/persuasion
  - erbacher/AmbigNQ-clarifying-question
  - tasksource/SIGA-nli
  - unigram/FOL-nli
  - tasksource/goal-step-wikihow
  - GGLab/PARADISE
  - tasksource/doc-nli
  - tasksource/mctest-nli
  - tasksource/patent-phrase-similarity
  - tasksource/natural-language-satisfiability
  - tasksource/idioms-nli
  - tasksource/lifecycle-entailment
  - nvidia/HelpSteer
  - nvidia/HelpSteer2
  - sadat2307/MSciNLI
  - pushpdeep/UltraFeedback-paired
  - tasksource/AES2-essay-scoring
  - tasksource/english-grading
  - tasksource/wice
  - Dzeniks/hover
  - sileod/missing-item-prediction
  - tasksource/tasksource_dpo_pairs
language:
  - en
  - zh
license: apache-2.0
library_name: transformers
tags:
  - multimodal
  - vqa
  - text
  - audio
metrics:
  - accuracy
  - bleu
  - wer
model-index:
  - name: Evolutionary Multi-Modal Model
    results:
      - task:
          type: vqa
          name: Visual Question Answering
        dataset:
          type: synthetic-dataset
          name: Synthetic Multimodal Dataset
          split: test
        metrics:
          - type: accuracy
            value: 85
pipeline_tag: text-generation
widget:
  - text: >-
      Is this review positive or negative? Review: Best cast iron skillet you
      will ever buy.
    example_title: Sentiment analysis
  - text: >-
      Barack Obama nominated Hilary Clinton as his secretary of state on Monday.
      He chose her because she had ...
    example_title: Coreference resolution
  - text: >-
      On a shelf, there are five books: a gray book, a red book, a purple book,
      a blue book, and a black book ...
    example_title: Logic puzzles
  - text: >-
      The two men running to become New York City's next mayor will face off in
      their first debate Wednesday night ...
    example_title: Reading comprehension

Model Sources

You need to use separate code, audio, text, and natural language together with the model. Because the model will use separate word segmenters and vocabularies to achieve the best results when dealing with special cases.

Multi-Modal Model

Model Card for Evolutionary

-- ###############################################

Model breast_cancer_wisconsin_original test 

from ucimlrepo import fetch_ucirepo 
fetch dataset 
breast_cancer_wisconsin_original = fetch_ucirepo(id=15) 
  
data (as pandas dataframes) 
X = breast_cancer_wisconsin_original.data.features 
y = breast_cancer_wisconsin_original.data.targets 
 
metadata 
print(breast_cancer_wisconsin_original.metadata) 
 
variable information 
print(breast_cancer_wisconsin_original.variables)

##########################################################

0 0.93 0.99 0.96 79

1 0.98 0.90 0.94 58

-- #accuracy 0.95 137

-- This model, named Evolutionary Multi-Modal Model, is a multimodal transformer designed to handle a variety of tasks including vision and audio processing. It is built on top of the adapter-transformers and transformers libraries and is intended to be a versatile base model for both direct use and fine-tuning.

-- Developed by: Independent researcher Funded by : Self-funded Shared by : Independent researcher Model type: Multimodal Language(s) (NLP): English zh License: Apache-2.0 Finetuned from model : None

Uses：https://huggingface.co/zeroMN/SHMT

Direct Use 

git lfs install

git clone https://huggingface.co/zeroMN/SHMT.git

Downstream Use

The model can be fine-tuned for specific tasks such as visual question answering (VQA), image captioning, and audio recognition.

Out-of-Scope Use

The Evolved Multimodal Model is not suitable for tasks that require high expertise or domain-specific expertise beyond its current capabilities. The number of speech frames still needs to be fine-tuned by yourself.

Bias, Risks, and Limitations

Recommendations

Users (both direct and downstream) should be made aware of the following risks, biases, and limitations:

  • Bias: The model may exhibit biases present in the training data, particularly if the data is not representative of all populations.
  • Risks: The model should not be used in critical applications where high accuracy and reliability are required without thorough testing and validation.
  • Limitations: The model may not perform well on tasks that require fine-grained recognition or highly specialized audio processing.

How to Get Started with the Model 

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zeroMN/SHMT")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("zeroMN/SHMT")