turkish-zeroshot-distilbert

This model is a fine-tuned version of dbmdz/distilbert-base-turkish-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7510
• Accuracy: 0.7201
• F1: 0.7207
• Precision: 0.7290
• Recall: 0.7201

Usage

# Use a pipeline as a high-level helper

pipe = pipeline(
    "zero-shot-classification",
    model="kaixkhazaki/turkish-zeroshot-distilbert",
    tokenizer="kaixkhazaki/turkish-zeroshot-distilbert",
    device=0 if torch.cuda.is_available() else -1  # Use GPU if available
)

#Enter your text and possible candidates of classification
sequence = "Bu laptopun pil ömrü ne kadar dayanıyor?"
candidate_labels = ["ürün özellikleri", "soru", "bilgi talebi", "laptop", "teknik destek"]

pipe(
    sequence,
    candidate_labels,
)

>>
{'sequence': 'Bu laptopun pil ömrü ne kadar dayanıyor?',
'labels': ['ürün özellikleri', 'laptop', 'soru', 'teknik destek', 'bilgi talebi'],
'scores': [0.4050311744213104, 0.1970272809267044, 0.1365433931350708, 0.13210774958133698, 0.1292904019355774]}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 32
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 500
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
1.0957	0.0326	200	1.0956	0.3506	0.2447	0.3575	0.3506
1.0092	0.0652	400	0.9754	0.5305	0.5296	0.5476	0.5305
0.9338	0.0978	600	0.8756	0.6080	0.6078	0.6098	0.6080
0.8987	0.1304	800	0.8632	0.6112	0.6107	0.6133	0.6112
0.9019	0.1630	1000	0.8275	0.6289	0.6291	0.6299	0.6289
0.8854	0.1956	1200	0.8219	0.6185	0.6184	0.6439	0.6185
0.8877	0.2282	1400	0.8108	0.6265	0.6249	0.6474	0.6265
0.8653	0.2608	1600	0.8147	0.6317	0.6320	0.6346	0.6317
0.8465	0.2934	1800	0.8109	0.6277	0.6269	0.6556	0.6277
0.8205	0.3259	2000	0.7946	0.6430	0.6423	0.6480	0.6430
0.8584	0.3585	2200	0.7998	0.6438	0.6414	0.6592	0.6438
0.8393	0.3911	2400	0.7971	0.6534	0.6536	0.6719	0.6534
0.8136	0.4237	2600	0.7695	0.6566	0.6562	0.6688	0.6566
0.8113	0.4563	2800	0.7614	0.6743	0.6739	0.6756	0.6743
0.8291	0.4889	3000	0.7589	0.6695	0.6704	0.6756	0.6695
0.8274	0.5215	3200	0.7591	0.6699	0.6697	0.6916	0.6699
0.8165	0.5541	3400	0.7379	0.6791	0.6795	0.6828	0.6791
0.7897	0.5867	3600	0.7467	0.6731	0.6734	0.6800	0.6731
0.79	0.6193	3800	0.7473	0.6679	0.6676	0.6804	0.6679
0.8108	0.6519	4000	0.7380	0.6687	0.6696	0.6786	0.6687
0.797	0.6845	4200	0.7429	0.6783	0.6791	0.6889	0.6783
0.7893	0.7171	4400	0.7405	0.6743	0.6747	0.6777	0.6743
0.7653	0.7497	4600	0.7551	0.6711	0.6708	0.6789	0.6711
0.772	0.7823	4800	0.7270	0.6859	0.6861	0.6941	0.6859
0.7686	0.8149	5000	0.7253	0.6819	0.6833	0.6909	0.6819
0.7431	0.8475	5200	0.7386	0.6731	0.6743	0.6926	0.6731
0.7968	0.8801	5400	0.7130	0.6936	0.6943	0.6978	0.6936
0.7584	0.9126	5600	0.7129	0.6960	0.6957	0.6973	0.6960
0.7629	0.9452	5800	0.7141	0.6827	0.6841	0.6983	0.6827
0.7477	0.9778	6000	0.7044	0.6920	0.6930	0.7029	0.6920
0.7043	1.0104	6200	0.7362	0.6880	0.6875	0.6996	0.6880
0.6868	1.0430	6400	0.7056	0.6972	0.6975	0.6999	0.6972
0.7048	1.0756	6600	0.7104	0.6896	0.6907	0.7016	0.6896
0.6965	1.1082	6800	0.7140	0.6988	0.6990	0.7021	0.6988
0.7043	1.1408	7000	0.7084	0.7028	0.7029	0.7109	0.7028
0.7111	1.1734	7200	0.6998	0.7008	0.7014	0.7066	0.7008
0.6994	1.2060	7400	0.7147	0.6964	0.6962	0.7043	0.6964
0.6992	1.2386	7600	0.6962	0.7028	0.7038	0.7129	0.7028
0.7161	1.2712	7800	0.7002	0.6964	0.6967	0.7039	0.6964
0.6935	1.3038	8000	0.7046	0.6972	0.6978	0.7079	0.6972
0.6858	1.3364	8200	0.7066	0.6996	0.7005	0.7144	0.6996
0.706	1.3690	8400	0.6956	0.7044	0.7053	0.7160	0.7044
0.7072	1.4016	8600	0.7158	0.6956	0.6953	0.7114	0.6956
0.6896	1.4342	8800	0.7090	0.6948	0.6952	0.7083	0.6948
0.6891	1.4668	9000	0.6936	0.6964	0.6977	0.7059	0.6964
0.6577	1.4993	9200	0.6926	0.7060	0.7072	0.7143	0.7060
0.6961	1.5319	9400	0.6792	0.7108	0.7106	0.7113	0.7108
0.6826	1.5645	9600	0.6843	0.7060	0.7066	0.7088	0.7060
0.695	1.5971	9800	0.6956	0.6896	0.6899	0.7013	0.6896
0.6904	1.6297	10000	0.7056	0.6948	0.6956	0.7030	0.6948
0.6982	1.6623	10200	0.6865	0.6988	0.6997	0.7041	0.6988
0.6723	1.6949	10400	0.7029	0.6932	0.6941	0.7105	0.6932
0.6658	1.7275	10600	0.6882	0.7060	0.7071	0.7122	0.7060
0.6929	1.7601	10800	0.6915	0.7028	0.7035	0.7139	0.7028
0.6742	1.7927	11000	0.6908	0.7044	0.7050	0.7171	0.7044
0.694	1.8253	11200	0.6960	0.7020	0.7021	0.7132	0.7020
0.6839	1.8579	11400	0.6894	0.7060	0.7069	0.7191	0.7060
0.682	1.8905	11600	0.6930	0.7020	0.7030	0.7161	0.7020
0.6806	1.9231	11800	0.6800	0.7112	0.7117	0.7182	0.7112
0.6936	1.9557	12000	0.6718	0.7076	0.7080	0.7143	0.7076
0.6917	1.9883	12200	0.6877	0.6972	0.6979	0.7088	0.6972
0.5941	2.0209	12400	0.6877	0.7161	0.7159	0.7168	0.7161
0.5729	2.0535	12600	0.7059	0.7120	0.7128	0.7165	0.7120
0.5849	2.0860	12800	0.7126	0.7084	0.7099	0.7181	0.7084
0.5937	2.1186	13000	0.6982	0.7137	0.7149	0.7220	0.7137
0.5975	2.1512	13200	0.7067	0.7048	0.7056	0.7143	0.7048
0.5877	2.1838	13400	0.7041	0.7088	0.7096	0.7124	0.7088
0.5801	2.2164	13600	0.7021	0.7185	0.7197	0.7249	0.7185
0.5897	2.2490	13800	0.7370	0.7012	0.7020	0.7160	0.7012
0.5986	2.2816	14000	0.6885	0.7173	0.7175	0.7211	0.7173
0.5702	2.3142	14200	0.6967	0.7201	0.7212	0.7251	0.7201
0.5885	2.3468	14400	0.6928	0.7084	0.7094	0.7173	0.7084
0.5955	2.3794	14600	0.6889	0.7165	0.7175	0.7222	0.7165
0.5981	2.4120	14800	0.6862	0.7193	0.7198	0.7258	0.7193
0.5974	2.4446	15000	0.6951	0.7165	0.7174	0.7244	0.7165
0.6057	2.4772	15200	0.6984	0.7108	0.7115	0.7199	0.7108
0.5939	2.5098	15400	0.7005	0.7169	0.7180	0.7248	0.7169
0.6026	2.5424	15600	0.7110	0.7120	0.7130	0.7213	0.7120
0.5794	2.5750	15800	0.7021	0.7213	0.7221	0.7285	0.7213
0.5743	2.6076	16000	0.6961	0.7157	0.7161	0.7222	0.7157
0.5987	2.6402	16200	0.6909	0.7201	0.7211	0.7258	0.7201
0.5741	2.6728	16400	0.7035	0.7084	0.7090	0.7163	0.7084
0.5628	2.7053	16600	0.7137	0.7068	0.7073	0.7210	0.7068
0.5632	2.7379	16800	0.7102	0.7084	0.7094	0.7270	0.7084
0.6049	2.7705	17000	0.6855	0.7181	0.7189	0.7274	0.7181
0.578	2.8031	17200	0.6946	0.7165	0.7172	0.7245	0.7165
0.5795	2.8357	17400	0.6919	0.7161	0.7169	0.7222	0.7161
0.5507	2.8683	17600	0.6898	0.7253	0.7260	0.7292	0.7253
0.5936	2.9009	17800	0.6892	0.7189	0.7197	0.7257	0.7189
0.5964	2.9335	18000	0.6826	0.7173	0.7182	0.7245	0.7173
0.5805	2.9661	18200	0.7005	0.7112	0.7124	0.7238	0.7112
0.6106	2.9987	18400	0.6886	0.7229	0.7236	0.7299	0.7229
0.4978	3.0313	18600	0.7325	0.7213	0.7218	0.7268	0.7213
0.5034	3.0639	18800	0.7586	0.7149	0.7158	0.7237	0.7149
0.4796	3.0965	19000	0.7483	0.7237	0.7242	0.7300	0.7237
0.5027	3.1291	19200	0.7195	0.7273	0.7282	0.7320	0.7273
0.4718	3.1617	19400	0.7576	0.7233	0.7239	0.7324	0.7233
0.4806	3.1943	19600	0.7427	0.7213	0.7219	0.7267	0.7213
0.4892	3.2269	19800	0.7586	0.7217	0.7222	0.7276	0.7217
0.4934	3.2595	20000	0.7593	0.7120	0.7128	0.7241	0.7120
0.4931	3.2920	20200	0.7459	0.7221	0.7228	0.7299	0.7221
0.4987	3.3246	20400	0.7301	0.7161	0.7168	0.7216	0.7161
0.4929	3.3572	20600	0.7499	0.7185	0.7193	0.7262	0.7185
0.4718	3.3898	20800	0.7398	0.7221	0.7228	0.7268	0.7221
0.4957	3.4224	21000	0.7343	0.7189	0.7197	0.7247	0.7189
0.496	3.4550	21200	0.7395	0.7141	0.7150	0.7231	0.7141
0.5113	3.4876	21400	0.7237	0.7213	0.7224	0.7287	0.7213
0.5009	3.5202	21600	0.7393	0.7205	0.7216	0.7276	0.7205
0.4793	3.5528	21800	0.7462	0.7217	0.7226	0.7278	0.7217
0.5007	3.5854	22000	0.7393	0.7229	0.7236	0.7284	0.7229
0.4836	3.6180	22200	0.7483	0.7173	0.7185	0.7275	0.7173
0.4885	3.6506	22400	0.7446	0.7201	0.7208	0.7285	0.7201
0.494	3.6832	22600	0.7368	0.7225	0.7235	0.7311	0.7225
0.476	3.7158	22800	0.7500	0.7165	0.7176	0.7278	0.7165
0.4787	3.7484	23000	0.7408	0.7201	0.7211	0.7281	0.7201
0.4983	3.7810	23200	0.7351	0.7181	0.7190	0.7265	0.7181
0.5081	3.8136	23400	0.7407	0.7197	0.7206	0.7287	0.7197
0.5209	3.8462	23600	0.7542	0.7137	0.7147	0.7248	0.7137
0.4924	3.8787	23800	0.7576	0.7169	0.7177	0.7280	0.7169
0.4939	3.9113	24000	0.7571	0.7161	0.7171	0.7258	0.7161
0.4792	3.9439	24200	0.7510	0.7201	0.7207	0.7290	0.7201

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0