Imbetten der Bildeinbettungen nicht erzeugen: Die Größe des Tensors A (1246) muss mit der Größe des Tensors B (77) in de ⇐ Python

[Anonymous]

Ich verwende ein Modell für Satztransformatoren, um Einbetten von Bilddateien (PIL-Image-Datei) zu erstellen. Es gibt jedoch den Fehler im Titel. Ich habe eine Reihe von Dingen versucht, um es zu lösen, aber ohne Erfolg. Ich denke, es kann eine einfache Lösung geben, aber ich kann sie nicht finden.

Code: Select all

import pandas as pd
from sentence_transformers import SentenceTransformer
import os
import numpy as np
from PIL import Image, ImageFile

ImageFile.LOAD_TRUNCATED_IMAGES = True

image_files = ['.jpg', '.jpeg', '.png']

class Analyzer:

def __init__(self):
self.image_model = SentenceTransformer("clip-ViT-B-32")

def analyze_directory(self, path):

files_data = []

with os.scandir(path) as dir_iter:
for entry in dir_iter:
try:
if entry.is_file():
_, ext = os.path.splitext(entry.name)
if ext in image_files:
try:
with Image.open(os.path.join(path, entry.name)) as img:
img.convert("RGB")
file_data = {
"Path": entry.name,
"Content": img,
"Type": "image"
}
except Exception as e:
file_data = {
"Path": entry.name,
"Content": "",
"Type": "image"
}

else:
file_data = {
"Path": entry.name,
"Content": "",
"Type": "unknown"
}

files_data.append(file_data)

except Exception as e:
continue

df = pd.DataFrame(files_data)

embeddings = []
for _, row in df.iterrows():
if row["Type"] == "image":
try:
img = img.resize((224, 224))
# Convert PIL Image to tensor
img_tensor = np.array(img)
# Normalize pixel values to [-1, 1] range expected by CLIP
img_normalized = (img_tensor / 255.0 * 2.0) - 1.0
img_batch = np.expand_dims(img_normalized, axis=0)
embedding = self.image_model.encode(str(img_batch)).numpy()[0]
except Exception as e:
raise RuntimeError(f"Failed to generate image embeddings: {str(e)}")
else:
# Handle unknown types
embedding = np.zeros(384)

embeddings.append(embedding)

embeddings = np.array(embeddings)

return embeddings
< /code>
Ich habe versucht, den Tensor abzuschneiden, aber ich konnte keinen Weg finden, dies zu tun.Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "C:\Users\...\src\document_analyzer\main.py", line 15, in 
main()
~~~~^^
File "C:\Users\...\src\document_analyzer\main.py", line 6, in main
folder_structure = analyzer.analyze_directory(path)
File "C:\Users\...\src\document_analyzer\analyzer.py", line 69, in analyze_directory
raise RuntimeError(f"Failed to generate image embeddings: {str(e)}")
RuntimeError: Failed to generate image embeddings: The size of tensor a (1203) must match the size of tensor b (77) at non-singleton dimension 1