Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pass through adapter when using minio (s3) #132

Merged
merged 4 commits into from
Feb 1, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 28 additions & 7 deletions lib/active_encode/engine_adapters/pass_through_adapter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,8 @@ class PassThroughAdapter
MEDIAINFO_PATH = ENV["MEDIAINFO_PATH"] || "mediainfo"
FFMPEG_PATH = ENV["FFMPEG_PATH"] || "ffmpeg"

def create(input_url, options = {})
# Decode file uris for ffmpeg (mediainfo works either way)
input_url = Addressable::URI.unencode(input_url) if input_url.starts_with? "file:///"
input_url = ActiveEncode.sanitize_input(input_url)
def create(original_input_url, options = {})
input_url = sanitize_input_url(original_input_url)

new_encode = ActiveEncode::Base.new(input_url, options)
new_encode.id = SecureRandom.uuid
Expand Down Expand Up @@ -76,9 +74,7 @@ def create(input_url, options = {})

# Copy derivatives to work directory
options[:outputs].each do |opt|
url = opt[:url]
output_path = working_path("outputs/#{ActiveEncode.sanitize_base opt[:url]}#{File.extname opt[:url]}", new_encode.id)
FileUtils.cp FileLocator.new(url).location, output_path
output_path = copy_derivative_to_working_path(opt[:url], new_encode.id)
filename_label_hash[output_path] = opt[:label]
end

Expand Down Expand Up @@ -271,6 +267,31 @@ def file_error(new_encode, input_url)
write_errors new_encode
new_encode
end

def sanitize_input_url(url)
input_url = if url.starts_with?("file://")
# Decode file uris for ffmpeg (mediainfo works either way)
Addressable::URI.unencode(url)
elsif url.starts_with?("s3://")
# Change s3 uris into presigned http urls
FileLocator.new(url).location
else
url
end
ActiveEncode.sanitize_input(input_url)
end

def copy_derivative_to_working_path(url, id)
output_path = working_path("outputs/#{ActiveEncode.sanitize_base url}#{File.extname url}", id)
if url.start_with? "s3://"
# Use aws-sdk-s3 download_file method
# Single request mode needed for compatibility with minio
FileLocator::S3File.new(url).object.download_file(output_path, mode: 'single_request')
else
FileUtils.cp FileLocator.new(url).location, output_path
end
output_path
end
end
end
end