CPU -Spike tritt beim Hochladen von Daten über SFTP oder Web -Benutzeroberfläche auf (lokale Datei oder über Kamera hoch ⇐ Python

Python-Programme
Post Reply Previous topicNext topic
Anonymous
 CPU -Spike tritt beim Hochladen von Daten über SFTP oder Web -Benutzeroberfläche auf (lokale Datei oder über Kamera hoch

Post by Anonymous »

Ich erwarte eine sehr niedrige CPU -Verwendung beim Hochladen von Daten über SFTP oder Web -Benutzeroberfläche in Minio. Aber CPU -Spike tritt beim Hochladen von Daten über SFTP oder Web UI auf. Code> Paket in Minio. (Data Lake)
  • Ref 1: https://min.io/docs/minio/linux/develop ... tocol.html (Minio ftp)
    Ich habe auch SSE und Komprimierung überprüft, aber sie sind deaktiviert.

    Code: Select all

    ubuntu@mycom:~$ mc admin config get admin compression
compression enable=off allow_encryption=off extensions=.txt,.log,.csv,.json,.tar,.xml,.bin mime_types=text/*,application/json,application/xml,binary/octet-stream

ubuntu@mycom:~$ mc admin config get admin kms
mc:  Unable to get server '[kms]' config: unknown subsystem: kms.

    Das Problem ist, dass die CPU -Verwendung drastisch steigt. [/b] Wenn es 10 SFTP gibt Verbindungen, der Server stirbt. />
  • 4core < /li>
    16GBI -Mem < /li>
    EBS -Typ -einzelner SSD -Laufwerk < /li>
    < /ul>
    (

    Code: Select all

    htop
    Bildschirm Wenn 5 SFTP -Verbindungen hergestellt werden)
    < Br /> (

    Code: Select all

    docker stats
    Bildschirm Wenn 5 SFTP -Verbindungen hergestellt werden)
    < Br />
    Ich habe den Minio-Service mit den Informationen docker-compose.yml unten ausgeführt.

    Code: Select all

      minio:
image: minio/minio:latest
container_name: myminio_dev
restart: always
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USERNAME}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
MINIO_PORT: ${MINIO_PORT}
MINIO_CONSOLE_PORT: ${MINIO_CONSOLE_PORT}
MINIO_SFTP_PORT: ${MINIO_SFTP_PORT}
command: server /data --console-address ":${MINIO_CONSOLE_PORT}" --sftp "address=:${MINIO_SFTP_PORT}" --sftp "ssh-private-key=${MINIO_SSH_PRIVATE_KEY}" --sftp "trusted-user-ca-key=${MINIO_TRUSTED_USER_CA_KEY}"
ports:
- "${MINIO_PORT}:${MINIO_PORT}"  # S3 API access port
- "${MINIO_CONSOLE_PORT}:${MINIO_CONSOLE_PORT}"  # MinIO console access port
- "${MINIO_FTP_PORT}:${MINIO_FTP_PORT}"  # MinIO FTP connection port
- "${MINIO_FTP_PASV_PORT}:${MINIO_FTP_PASV_PORT}"   # MinIO FTP Data ports
- "${MINIO_SFTP_PORT}:${MINIO_SFTP_PORT}"  # MinIO SFTP connection port
volumes:
- /mnt/minio:/data
- ${MINIO_CERT_PATH}:${MINIO_CERT_PATH}:ro
< /code>
[list]
mc admin trace --verbose myminio

Code: Select all

127.0.0.1:9000 [REQUEST s3.GetBucketLocation] [2025-02-12T02:21:27.719] [Client IP: 111.111.111.111]
127.0.0.1:9000 GET /device/?location=
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=7d41dce67a4751f47c5d0599480ec32c41dbff66d0d8fe691915d2466929e2da
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 User-Agent: MinIO (linux; amd64) minio-go/v7.0.83
127.0.0.1:9000 X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.721] [ Duration 2.03ms TTFB 2.013522ms ↑ 93 B  ↓ 142 B ]
127.0.0.1:9000 200 OK
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 X-Xss-Protection: 1; mode=block
127.0.0.1:9000 Content-Type: application/xml
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Content-Length: 142
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 X-Amz-Request-Id: 18235496CACB8D29
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 
ap-northeast-2
127.0.0.1:9000
127.0.0.1:9000 [REQUEST s3.HeadObject] [2025-02-12T02:21:27.721] [Client IP: 111.111.111.111]
127.0.0.1:9000 HEAD /device/6d60e92a-3e54-4ced-879c-e93423619d32
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/ap-northeast-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=9b6a05f795e981fe19f8f9378a8b25a8ec23fae15e05dde681b23eed15798b5c
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 User-Agent: MinIO (linux;  amd64) minio-go/v7.0.83
127.0.0.1:9000
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.721] [ Duration 279µs TTFB 254.422µs ↑ 93 B  ↓ 0 B ]
127.0.0.1:9000 404 Not Found
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 X-Minio-Error-Desc: "The specified key does not exist."
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 X-Minio-Error-Code: NoSuchKey
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 X-Amz-Request-Id: 18235496CAEF1FB2
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 X-Xss-Protection: 1; mode=block
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:9000
127.0.0.1:9000
127.0.0.1:9000 [REQUEST s3.GetBucketLocation] [2025-02-12T02:21:27.755] [Client IP: 111.111.111.111]
127.0.0.1:9000 GET /device/?location=
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 User-Agent: MinIO (linux; amd64) minio-go/v7.0.83
127.0.0.1:9000 X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=7d41dce67a4751f47c5d0599480ec32c41dbff66d0d8fe691915d2466929e2da
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.756] [ Duration 193µs TTFB 182.706µs ↑ 93 B  ↓ 142 B ]
127.0.0.1:9000 200 OK
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 X-Xss-Protection: 1; mode=block
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000 X-Amz-Request-Id: 18235496CCF9E232
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Content-Length: 142
127.0.0.1:9000 Content-Type: application/xml
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 
ap-northeast-2
127.0.0.1:9000
127.0.0.1:9000 [REQUEST s3.HeadBucket] [2025-02-12T02:21:27.756] [Client IP: 111.111.111.111]
127.0.0.1:9000 HEAD /device/
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/ap-northeast-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=ab3abf07a2b4380e10aa2fefad10f2fcec67b77fecfacfbfc5c0086ce70c37a3
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 User-Agent: MinIO (linux; amd64) minio-go/v7.0.83
127.0.0.1:9000 X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.756] [ Duration 346µs TTFB 303.346µs ↑ 93 B  ↓ 0 B ]
127.0.0.1:9000 200 OK
127.0.0.1:9000 X-Amz-Request-Id: 18235496CD02C67A
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 X-Xss-Protection: 1; mode=block
127.0.0.1:9000 Content-Type: application/xml
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000;  includeSubDomains
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000
127.0.0.1:9000
127.0.0.1:9000 [REQUEST s3.NewMultipartUpload] [2025-02-12T02:21:27.757] [Client IP: 111.111.111.111]
127.0.0.1:9000 POST /device/6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108%2B0000_15047.jpg?uploads=
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 User-Agent: MinIO (linux; amd64) minio-go/v7.0.83
127.0.0.1:9000 X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/ap-northeast-2/s3/aws4_request, SignedHeaders=content-type;host;x-amz-checksum-algorithm;x-amz-content-sha256;x-amz-date, Signature=805c0ee0e95dec65809396cd95687b7a24dc7f30d5460345b71c2231a8226ba0
127.0.0.1:9000 Content-Type: image/jpeg
127.0.0.1:9000 X-Amz-Checksum-Algorithm: CRC32C
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.763] [ Duration 6.011ms TTFB 5.997517ms ↑ 131 B  ↓ 425 B ]
127.0.0.1:9000 200 OK
127.0.0.1:9000 X-Amz-Request-Id: 18235496CD102674
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 X-Amz-Checksum-Algorithm: CRC32C
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000 X-Xss-Protection: 1; mode=block
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:9000 Content-Type: application/xml
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 Content-Length: 425
127.0.0.1:9000 
device6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108+0000_15047.jpgNWQ2ZmM0MWItY2JiNi00MTlmLWE2MGQtNjc1ZWZlODNjMDZkLjU2Y2NlMjc5LWEzNzYtNGI2My1iZWJiLWVhYjNhYjM3ZmI2ZngxNzM5MzI2ODg3NzU3NjEzOTY4
127.0.0.1:9000
127.0.0.1:9000 [REQUEST s3.PutObjectPart] [2025-02-12T02:21:27.825] [Client IP: 111.111.111.111]
127.0.0.1:9000 PUT /device/6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108%2B0000_15047.jpg?partNumber=1&uploadId=NWQ2ZmM0MWItY2JiNi00MTlmLWE2MGQtNjc1ZWZlODNjMDZkLjU2Y2NlMjc5LWEzNzYtNGI2My1iZWJiLWVhYjNhYjM3ZmI2ZngxNzM5MzI2ODg3NzU3NjEzOTY4
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/ap-northeast-2/s3/aws4_request, SignedHeaders=host;x-amz-checksum-crc32c;x-amz-content-sha256;x-amz-date, Signature=a0b1916ff74f5d55be19efdc040d1c714240cce329bc903f853cc191434bbf87
127.0.0.1:9000 Content-Length: 63971
127.0.0.1:9000 User-Agent: MinIO (linux; amd64) minio-go/v7.0.83
127.0.0.1:9000 X-Amz-Checksum-Crc32c: soz+Pw==
127.0.0.1:9000 X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000 
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.835] [ Duration 9.563ms TTFB 9.532614ms ↑ 63 KiB  ↓ 0 B ]
127.0.0.1:9000 200 OK
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 X-Amz-Request-Id: 18235496D125361C
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 X-Xss-Protection: 1;  mode=block
127.0.0.1:9000 ETag: "73772167823c98517f206e8516eee282"
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Checksum-Crc32c: soz+Pw==
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Content-Length: 0
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 
127.0.0.1:9000
127.0.0.1:9000 [REQUEST s3.CompleteMultipartUpload] [2025-02-12T02:21:27.836] [Client IP: 111.111.111.111]
127.0.0.1:9000 POST /device/6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108%2B0000_15047.jpg?uploadId=NWQ2ZmM0MWItY2JiNi00MTlmLWE2MGQtNjc1ZWZlODNjMDZkLjU2Y2NlMjc5LWEzNzYtNGI2My1iZWJiLWVhYjNhYjM3ZmI2ZngxNzM5MzI2ODg3NzU3NjEzOTY4
127.0.0.1:9000 Proto: HTTP/1.1
127.0.0.1:9000 Host: 127.0.0.1:9000
127.0.0.1:9000 Authorization: AWS4-HMAC-SHA256 Credential=6d60e92a-3e54-4ced-879c-e93423619d32/20250212/ap-northeast-2/s3/aws4_request, SignedHeaders=content-type;host;x-amz-checksum-crc32c;x-amz-content-sha256;x-amz-date, Signature=f6a3f50afac7da32754cad037fa555cdd82af9989cdf3b2e6a5b55fea5291545
127.0.0.1:9000 Content-Length: 224
127.0.0.1:9000 Content-Type: application/octet-stream
127.0.0.1:9000 User-Agent: MinIO (linux; amd64) minio-go/v7.0.83
127.0.0.1:9000 X-Amz-Checksum-Crc32c: aPggMg==
127.0.0.1:9000 X-Amz-Content-Sha256: c268af49ab2e6b9d4335e8dd3ac574f6642f3c5169b994f741b5740c814c8183
127.0.0.1:9000 X-Amz-Date: 20250212T022127Z
127.0.0.1:9000 X-Forwarded-For: 111.111.111.111
127.0.0.1:9000 173772167823c98517f206e8516eee282soz+Pw==
127.0.0.1:9000 [RESPONSE] [2025-02-12T02:21:27.840] [ Duration 4.309ms TTFB 4.2764ms ↑ 352 B  ↓ 526 B ]
127.0.0.1:9000 200 OK
127.0.0.1:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:9000 Vary: Origin,Accept-Encoding
127.0.0.1:9000 X-Amz-Request-Id: 18235496D1C12C9C
127.0.0.1:9000 X-Xss-Protection: 1; mode=block
127.0.0.1:9000 ETag: "88128f2e7759186315b86c0b1bef696b-1"
127.0.0.1:9000 x-amz-version-id: b56e6fdc-0881-4bc6-a89c-d8a82c63a3cd
127.0.0.1:9000 Content-Type: application/xml
127.0.0.1:9000 X-Amz-Checksum-Crc32c: aPggMg==-1
127.0.0.1:9000 X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
127.0.0.1:9000 X-Content-Type-Options: nosniff
127.0.0.1:9000 X-Ratelimit-Limit: 3283
127.0.0.1:9000 Accept-Ranges: bytes
127.0.0.1:9000 Content-Length: 526
127.0.0.1:9000 Server: MinIO
127.0.0.1:9000 X-Amz-Bucket-Region: ap-northeast-2
127.0.0.1:9000 X-Ratelimit-Remaining: 3283
127.0.0.1:9000 
http://127.0.0.1:9000/device/6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108+0000_15047.jpgdevice6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108+0000_15047.jpg"88128f2e7759186315b86c0b1bef696b-1"aPggMg==-1
127.0.0.1:9000
< /code>
[list]
[*]mc admin trace -call=ftp --verbose myminio
[/list]

Code: Select all

127.0.0.1:9000  [FTP Stat] [2025-02-12T02:24:05.246] /device/6d60e92a-3e54-4ced-879c-e93423619d32 cmd=Stat param=/device/6d60e92a-3e54-4ced-879c-e93423619d32 source=[sftp-server-driver.go:397:(*sftpDriver).Filelist()] user=6d60e92a-3e54-4ced-879c-e93423619d32 1.477451ms
127.0.0.1:9000  [FTP Put] [2025-02-12T02:24:05.256] /device/6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108+0000_15047.jpg user=6d60e92a-3e54-4ced-879c-e93423619d32 cmd=Put param=/device/6d60e92a-3e54-4ced-879c-e93423619d32/GBR_NorthEastLive_GHCBSA1-01_20240909T121108+0000_15047.jpg source=[sftp-server-driver.go:248:(*sftpDriver).Filewrite()] 78.230517ms 62 KiB
< /code>
[list]
[*]uploader.py
Code
[/list]
class SFTPClient(Collector):
def __init__(self, response, args, **kwargs):
super().__init__()

self.conn = None
self.response = dict(response)
self.args = args

keylist = ['hostname', 'port', 'username', 'password',
'path', 'private_key_path', 'passphrase',
'bucket_name', 'device_uuid', 'datetime']

for key in keylist:
setattr(self, key, self.response.get(key, None))

self.upload = self.args.upload
self.mode = self.args.mode
self.interval = self.args.interval
self.cam_id = self.args.cam_id

if self.args.private_key_path is not None:
logging.info(f"Update `private_key_path`: {self.private_key_path} -> {self.args.private_key_path}")
self.private_key_path = self.args.private_key_path

if self.args.passphrase is not None:
logging.info(f"Update `passphrase`: {self.passphrase if self.passphrase is not None else None} -> {'*' * len(self.args.passphrase)}")
self.passphrase = self.args.passphrase

logging.info(f"Arguments: {self.args}")

for key, val in kwargs.items():
if key not in keylist:
logging.error(f"Invalid key: `{key}`")
return
setattr(self, key, val)

if self.mode == 'local':
logging.info(f"Upload path: {self.upload}")
elif self.mode == 'camera':
logging.info(f"Camera id: {self.cam_id}(interval: {self.interval}")

async def _create_client(self):
try:
'''
if self.passphrase:
private_key = asyncssh.read_private_key(self.private_key_path, password=self.passphrase)
else:
private_key = asyncssh.read_private_key(self.private_key_path)
'''

self.conn = await asyncssh.connect(
host=self.hostname,
port=int(self.port),
username=self.username,
client_keys=[self.private_key_path],
passphrase=self.passphrase,
known_hosts=None
)

logging.info(f"SSH connection established to {self.hostname}:{self.port} as {self.username}.")
logging.info(f"connection: {self.conn} .")

except (OSError, asyncssh.Error) as exc:
logging.error(f"SSH/SFTP connection failed: {exc}")
traceback.print_exc()

return self.conn

async def upload_from_local(self):
is_file = True
if not self.conn:
self.conn = await self._create_client()

if not self.conn:
logging.error("Failed to create SSH connection. Aborting upload.")
return

try:
async with self.conn.start_sftp_client() as sftp:
logging.info(f"SFTP session opened: {sftp} .")

if not os.path.isfile(self.upload): # is not file
if os.path.isdir(self.upload): # is directory
object_list = glob(os.path.join(self.upload, f"*"), recursive=True)
self.upload = [x for x in object_list if os.path.isfile(x)]
else: # is not file and not directory
raise NotImplementedError
else: # is file
func = sftp.put

logging.info(f"Uploading '{self.upload}' to '{self.path}'...")

if isinstance(self.upload, list):
for targetf in self.upload:
if targetf is None:
continue
else:
await sftp.put(targetf, self.path, block_size=32768)
# await asyncio.sleep(self.interval)
elif os.path.isfile(self.upload):
await sftp.put(self.upload, self.path, block_size=32768)

# await func(self.upload, self.path, block_size=4096)
result = {"code": 200, "message": "success", "data": {"src": self.upload, "dst": self.path}}
logging.info(json.dumps(result))

except (OSError, asyncssh.Error, AttributeError) as failed_exc:
logging.error(f"SSH/SFTP connection failed: {failed_exc}")
traceback.print_exc()
except asyncssh.sftp.SFTPNoSuchFile as notarget_exc:
logging.error(f"SSH/SFTP cannot found file/path to upload: {notarget_exc}")
traceback.print_exc()
except asyncio.CancelledError as cancelled_exc:
logging.error(f"Upload task cancelled: {cancelled_exc}")
traceback.print_exc()
except TimeoutError as timeout_exc:
logging.error(f"Timeout: {timeout_exc}")
traceback.print_exc()
except Exception as e:
logging.error(f"An unexpected error occurred: {e}")
traceback.print_exc()

finally:
if self.conn:
self.conn.close()
await self.conn.wait_closed()
self.conn = None
logging.info("SSH connection closed.")
< /code>
Ich arbeite zum ersten Mal an SFTP und es ist nicht einfach. Bitte hilf mir. Was soll ich noch überprüfen?
Top
Post Reply Previous topicNext topic

Quick Reply

Change Text Case: 
   
  • Similar Topics
    Replies
    Views
    Last post

Return to “Python”