Skip to content

Commit 9ff77dd

Browse files
authored
Merge pull request #161 from zzq996/master
Version 3.24.8
2 parents 5af1b31 + ab7c0f8 commit 9ff77dd

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+2402
-153
lines changed

README-Android.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
1+
Version 3.24.8
2+
Resolved issues:
3+
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart now supports crc64 checksum.
4+
2. UploadFile can be cancelled and aborted now.
5+
3. Allow you set okhttp's EventListenerFactory to profile each stage of a http request,not set by default.
6+
4. Fixed the issue that client encryption is available only in obs protocal and add some check logic when encryption algrithm is null.
7+
5. Optimised the logic of set progress listener when using uploadFile.
8+
6. Optimised some log info format.
9+
7. Added some logic to compatible with Android 7.0 when using DateTimeFormatter.
10+
11+
Third-party dependence:
12+
1. Replace okio 3.8.0 with okio 3.6.0
13+
2. Replace log4j-core 2.20.0 with log4j-core 2.18.0
14+
3. Replace jackson-core 2.15.4 with jackson-core 2.15.2
15+
4. Replace jackson-databind 2.15.4 with jackson-databind 2.15.2
16+
5. Replace jackson-annotations 2.15.4 with jackson-annotations 2.15.2
17+
6. Replace log4j-api 2.20.0 with log4j-api 2.18.0
18+
-----------------------------------------------------------------------------------
119
Version 3.24.3
220
Resolved issues:
321
1. Optimized log info of some exception stack

README-Java.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
1+
Version 3.24.8
2+
Resolved issues:
3+
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart now supports crc64 checksum.
4+
2. UploadFile can be cancelled and aborted now.
5+
3. Allow you set okhttp's EventListenerFactory to profile each stage of a http request,not set by default.
6+
4. Fixed the issue that client encryption is available only in obs protocal and add some check logic when encryption algrithm is null.
7+
5. Optimised the logic of set progress listener when using uploadFile.
8+
6. Optimised some log info format.
9+
7. Added some logic to compatible with Android 7.0 when using DateTimeFormatter.
10+
11+
Third-party dependence:
12+
1. Replace okio 3.8.0 with okio 3.6.0
13+
2. Replace log4j-core 2.20.0 with log4j-core 2.18.0
14+
3. Replace jackson-core 2.15.4 with jackson-core 2.15.2
15+
4. Replace jackson-databind 2.15.4 with jackson-databind 2.15.2
16+
5. Replace jackson-annotations 2.15.4 with jackson-annotations 2.15.2
17+
6. Replace log4j-api 2.20.0 with log4j-api 2.18.0
18+
-----------------------------------------------------------------------------------
119
Version 3.24.3
220
Resolved issues:
321
1. Optimized log info of some exception stack

README.MD

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
1+
Version 3.24.8
2+
Resolved issues:
3+
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart now supports crc64 checksum.
4+
2. UploadFile can be cancelled and aborted now.
5+
3. Allow you set okhttp's EventListenerFactory to profile each stage of a http request,not set by default.
6+
4. Fixed the issue that client encryption is available only in obs protocal and add some check logic when encryption algrithm is null.
7+
5. Optimised the logic of set progress listener when using uploadFile.
8+
6. Optimised some log info format.
9+
7. Added some logic to compatible with Android 7.0 when using DateTimeFormatter.
10+
11+
Third-party dependence:
12+
1. Replace okio 3.8.0 with okio 3.6.0
13+
2. Replace log4j-core 2.20.0 with log4j-core 2.18.0
14+
3. Replace jackson-core 2.15.4 with jackson-core 2.15.2
15+
4. Replace jackson-databind 2.15.4 with jackson-databind 2.15.2
16+
5. Replace jackson-annotations 2.15.4 with jackson-annotations 2.15.2
17+
6. Replace log4j-api 2.20.0 with log4j-api 2.18.0
18+
-----------------------------------------------------------------------------------
119
Version 3.24.3
220
Resolved issues:
321
1. Optimized log info of some exception stack

README_CN.MD

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
1+
Version 3.24.8
2+
Resolved issues:
3+
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart支持crc64校验
4+
2. 断点续传上传支持暂停、取消
5+
3. 支持设置okhttp的EventListenerFactory,用于统计http请求各阶段耗时,默认关闭
6+
4. 修复客户端加密只能在obs协议下使用的问题,增加加密算法为null时的判断
7+
5. 优化断点续传上传时的进度条设置逻辑
8+
6. 优化部分日志打印格式
9+
7. 使用DateTimeFormatter时兼容Android 7.0
10+
11+
Third-party dependence:
12+
1. 使用 okio 3.8.0 替代 okio 3.6.0
13+
2. 使用 log4j-core 2.20.0 替代 log4j-core 2.18.0
14+
3. 使用 jackson-core 2.15.4 替代 jackson-core 2.15.2
15+
4. 使用 jackson-databind 2.15.4 替代 jackson-databind 2.15.2
16+
5. 使用 jackson-annotations 2.15.4 替代 jackson-annotations 2.15.2
17+
6. 使用 log4j-api 2.20.0 替代 log4j-api 2.18.0
18+
-----------------------------------------------------------------------------------
119
Version 3.24.3
220
Resolved issues:
321
1. 优化某些堆栈的日志打印

app/src/main/java/com/obs/log/LoggerBuilder.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,8 @@ static class GetLoggerHolder {
3939
try {
4040
loggerClass = Class.forName("java.util.logging.Logger");
4141
getLoggerClass = GetLoggerHolder.loggerClass.getMethod("getLogger", String.class);
42-
} catch (NoSuchMethodException | SecurityException | ClassNotFoundException |
43-
NoClassDefFoundError exx) {
42+
} catch (NoSuchMethodException | SecurityException | ClassNotFoundException
43+
| NoClassDefFoundError exx) {
4444
ILOG.warning(exx.getMessage());
4545
}
4646
}

app/src/main/java/com/obs/services/AbstractClient.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,8 @@ protected void init(String accessKey, String secretKey, String securityToken, Ob
6868
if (this.isAuthTypeNegotiation()) {
6969
this.getProviderCredentials().setIsAuthTypeNegotiation(true);
7070
}
71-
this.initHttpClient(config.getHttpDispatcher(), config.getCustomizedDnsImpl(), config.getHostnameVerifier());
71+
this.initHttpClient(config.getHttpDispatcher(), config.getCustomizedDnsImpl(), config.getHostnameVerifier(),
72+
config.getEventListenerFactory());
7273
OBSXMLBuilder.setXmlDocumentBuilderFactoryClass(config.getXmlDocumentBuilderFactoryClass());
7374
reqBean.setRespTime(new Date());
7475
reqBean.setResultCode(Constants.RESULTCODE_SUCCESS);
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
package com.obs.services;
2+
3+
import com.obs.services.internal.task.UploadFileTask;
4+
import com.obs.services.model.CompleteMultipartUploadResult;
5+
import com.obs.services.model.TaskCallback;
6+
import com.obs.services.model.UploadFileRequest;
7+
8+
public interface IObsClientAsync {
9+
UploadFileTask uploadFileAsync(
10+
UploadFileRequest uploadFileRequest,
11+
TaskCallback<CompleteMultipartUploadResult, UploadFileRequest> completeCallback);
12+
}
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
package com.obs.services;
2+
3+
import com.obs.log.ILogger;
4+
import com.obs.log.LoggerBuilder;
5+
import com.obs.services.internal.task.UploadFileTask;
6+
import com.obs.services.model.CompleteMultipartUploadResult;
7+
import com.obs.services.model.TaskCallback;
8+
import com.obs.services.model.UploadFileRequest;
9+
10+
import java.io.IOException;
11+
import java.util.concurrent.Callable;
12+
import java.util.concurrent.ExecutorService;
13+
import java.util.concurrent.Executors;
14+
import java.util.concurrent.Future;
15+
16+
public class ObsClientAsync extends ObsClient implements IObsClientAsync {
17+
/**
18+
* Constructor
19+
*
20+
* @param endPoint OBS endpoint
21+
*/
22+
public ObsClientAsync(String endPoint) {
23+
super(endPoint);
24+
}
25+
26+
/**
27+
* Constructor
28+
*
29+
* @param config Configuration parameters of ObsClient
30+
*/
31+
public ObsClientAsync(ObsConfiguration config) {
32+
super(config);
33+
}
34+
35+
/**
36+
* Constructor
37+
*
38+
* @param accessKey AK in the access key
39+
* @param secretKey SK in the access key
40+
* @param endPoint OBS endpoint
41+
*/
42+
public ObsClientAsync(String accessKey, String secretKey, String endPoint) {
43+
super(accessKey, secretKey, endPoint);
44+
}
45+
46+
/**
47+
* Constructor
48+
*
49+
* @param accessKey AK in the access key
50+
* @param secretKey SK in the access key
51+
* @param config Configuration parameters of ObsClient
52+
*/
53+
public ObsClientAsync(String accessKey, String secretKey, ObsConfiguration config) {
54+
super(accessKey, secretKey, config);
55+
}
56+
57+
/**
58+
* Constructor
59+
*
60+
* @param accessKey AK in the temporary access key
61+
* @param secretKey SK in the temporary access key
62+
* @param securityToken Security token
63+
* @param endPoint OBS endpoint
64+
*/
65+
public ObsClientAsync(String accessKey, String secretKey, String securityToken, String endPoint) {
66+
super(accessKey, secretKey, securityToken, endPoint);
67+
}
68+
69+
/**
70+
* Constructor
71+
*
72+
* @param accessKey AK in the temporary access key
73+
* @param secretKey SK in the temporary access key
74+
* @param securityToken Security token
75+
* @param config Configuration parameters of ObsClient
76+
*/
77+
public ObsClientAsync(String accessKey, String secretKey, String securityToken, ObsConfiguration config) {
78+
super(accessKey, secretKey, securityToken, config);
79+
}
80+
81+
public ObsClientAsync(IObsCredentialsProvider provider, String endPoint) {
82+
super(provider, endPoint);
83+
}
84+
85+
public ObsClientAsync(IObsCredentialsProvider provider, ObsConfiguration config) {
86+
super(provider, config);
87+
}
88+
89+
private static final ILogger log = LoggerBuilder.getLogger(ObsClientAsync.class);
90+
private ExecutorService asyncClientExecutorService;
91+
private static final int DEFAULT_CLIENT_EXECUTOR_SERVICE_SIZE = 128;
92+
93+
private int queryInterval = 1000;
94+
95+
@Override
96+
public void close() throws IOException {
97+
log.warn("ObsClientAsync closing");
98+
try {
99+
// finishing all task
100+
getExecutorService().shutdown();
101+
log.warn("ObsClientAsync closed");
102+
} catch (Exception e) {
103+
log.warn("ObsClientAsync close failed, detail:", e);
104+
}
105+
super.close();
106+
}
107+
108+
private static final String ASYNC_CLIENT_EXECUTOR_SERVICE_THREAD_NAME = "async-client-thread";
109+
protected ExecutorService getExecutorService() {
110+
if (asyncClientExecutorService == null) {
111+
asyncClientExecutorService = Executors.newFixedThreadPool(DEFAULT_CLIENT_EXECUTOR_SERVICE_SIZE,
112+
r -> new Thread(r, ASYNC_CLIENT_EXECUTOR_SERVICE_THREAD_NAME));
113+
}
114+
return asyncClientExecutorService;
115+
}
116+
117+
public void setExecutorService(ExecutorService service) {
118+
if (asyncClientExecutorService != null) { // wait for all task finish
119+
asyncClientExecutorService.shutdown();
120+
while (!asyncClientExecutorService.isTerminated()) {
121+
try {
122+
Thread.sleep(queryInterval);
123+
} catch (InterruptedException e) {
124+
log.warn("ObsClientAsync setExecutorService failed, detail:", e);
125+
}
126+
}
127+
}
128+
asyncClientExecutorService = service;
129+
}
130+
131+
public int getQueryInterval() {
132+
return queryInterval;
133+
}
134+
135+
public void setQueryInterval(int queryInterval) {
136+
this.queryInterval = queryInterval;
137+
}
138+
139+
/**
140+
* @param uploadFileRequest
141+
* @param completeCallback
142+
* @return
143+
*/
144+
@Override
145+
public UploadFileTask uploadFileAsync(
146+
UploadFileRequest uploadFileRequest,
147+
TaskCallback<CompleteMultipartUploadResult, UploadFileRequest> completeCallback) {
148+
log.debug("start uploadFileAsync");
149+
if (uploadFileRequest.getCancelHandler() != null) {
150+
uploadFileRequest.getCancelHandler().resetCancelStatus();
151+
}
152+
UploadFileTask uploadFileTask =
153+
new UploadFileTask(this, uploadFileRequest.getBucketName(), uploadFileRequest, completeCallback);
154+
Future<?> future = getExecutorService().submit((Callable<?>) uploadFileTask);
155+
156+
uploadFileTask.setResultFuture(future);
157+
return uploadFileTask;
158+
}
159+
}

app/src/main/java/com/obs/services/ObsConfiguration.java

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424

2525
import okhttp3.Dispatcher;
2626
import okhttp3.Dns;
27+
import okhttp3.EventListener;
2728

2829
import java.security.SecureRandom;
2930

@@ -104,6 +105,7 @@ public class ObsConfiguration implements Cloneable {
104105
private HostnameVerifier hostnameVerifier;
105106

106107
private String xmlDocumentBuilderFactoryClass;
108+
private EventListener.Factory eventListenerFactory;
107109

108110
/**
109111
* Constructor
@@ -939,4 +941,15 @@ public SecureRandom getSecureRandom() {
939941
public void setSecureRandom(SecureRandom secureRandom) {
940942
this.secureRandom = secureRandom;
941943
}
944+
945+
public EventListener.Factory getEventListenerFactory()
946+
{
947+
return eventListenerFactory;
948+
}
949+
950+
public void setEventListenerFactory(EventListener.Factory eventListenerFactory)
951+
{
952+
this.eventListenerFactory = eventListenerFactory;
953+
}
954+
942955
}

app/src/main/java/com/obs/services/crypto/CryptoObsClient.java

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -339,16 +339,20 @@ protected ObsObject getObjectImpl(GetObjectRequest request) throws ServiceExcept
339339
// 该接口是下载对象,需要将流返回给客户(调用方),我们不能关闭这个流
340340

341341
if (ctrCipherGenerator != null) {
342+
String headerMetaPrefix =
343+
this.getProviderCredentials() != null &&
344+
this.getProviderCredentials().getLocalAuthType(request.getBucketName()) != AuthTypeEnum.OBS
345+
? Constants.V2_HEADER_META_PREFIX : Constants.OBS_HEADER_META_PREFIX;
342346
String encryptedAlgorithm =
343347
(String)
344348
objMetadata
345349
.getOriginalHeaders()
346-
.get(Constants.OBS_HEADER_META_PREFIX + ENCRYPTED_ALGORITHM_META_NAME);
350+
.get(headerMetaPrefix + ENCRYPTED_ALGORITHM_META_NAME);
347351
String encryptedStart =
348352
(String)
349353
objMetadata
350354
.getOriginalHeaders()
351-
.get(Constants.OBS_HEADER_META_PREFIX + ENCRYPTED_START_META_NAME);
355+
.get(headerMetaPrefix + ENCRYPTED_START_META_NAME);
352356

353357
if (isValidEncryptedAlgorithm(encryptedAlgorithm)) {
354358
byte[] cryptoKeyBytes = ctrCipherGenerator.getCryptoKeyBytes();
@@ -365,7 +369,7 @@ protected ObsObject getObjectImpl(GetObjectRequest request) throws ServiceExcept
365369
objMetadata
366370
.getOriginalHeaders()
367371
.get(
368-
Constants.OBS_HEADER_META_PREFIX
372+
headerMetaPrefix
369373
+ ENCRYPTED_AES_KEY_META_NAME);
370374
// 解密rsa加密后的主密钥
371375
cryptoKeyBytes =
@@ -425,8 +429,8 @@ protected ObsObject getObjectImpl(GetObjectRequest request) throws ServiceExcept
425429
}
426430

427431
public boolean isValidEncryptedAlgorithm(String encryptedAlgorithm) {
428-
return encryptedAlgorithm.equals(CtrRSACipherGenerator.ENCRYPTED_ALGORITHM)
429-
|| encryptedAlgorithm.equals(CTRCipherGenerator.ENCRYPTED_ALGORITHM);
432+
return encryptedAlgorithm != null && (encryptedAlgorithm.equals(CtrRSACipherGenerator.ENCRYPTED_ALGORITHM)
433+
|| encryptedAlgorithm.equals(CTRCipherGenerator.ENCRYPTED_ALGORITHM));
430434
}
431435

432436
protected byte[] getOrGenerateCryptoIvBytes() {

0 commit comments

Comments
 (0)