Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Optimize the logic of obtaining browser data. #310

Open
1 task
moonD4rk opened this issue Jan 30, 2024 · 2 comments
Open
1 task

Feature Request: Optimize the logic of obtaining browser data. #310

moonD4rk opened this issue Jan 30, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@moonD4rk
Copy link
Owner

Feature Description

The current way of obtaining browser data is to traverse the folder through filepath.walk, obtain the entire directory, and finally return the corresponding browser data type (item) and corresponding file path.
This method has low efficiency and the data structure is not intuitive. This part needs to be optimized.

func chromiumWalkFunc(items []item.Item, multiItemPaths map[string]map[item.Item]string) filepath.WalkFunc {
	return func(path string, info fs.FileInfo, err error) error {
		for _, v := range items {
			if info.Name() != v.Filename() {
				continue
			}
			if strings.Contains(path, "System Profile") {
				continue
			}
			profileFolder := fileutil.ParentBaseDir(path)
			if strings.Contains(filepath.ToSlash(path), "/Network/Cookies") {
				profileFolder = fileutil.BaseDir(strings.ReplaceAll(filepath.ToSlash(path), "/Network/Cookies", ""))
			}
			if _, exist := multiItemPaths[profileFolder]; exist {
				multiItemPaths[profileFolder][v] = path
			} else {
				multiItemPaths[profileFolder] = map[item.Item]string{v: path}
			}
		}
		return err
	}
}

Why is this feature needed?

Improve code readability and increase file retrieval efficiency.

Checklist

Screenshots/Videos

If applicable, add screenshots or videos to help explain your proposal.

Additional Context

Add any other context or screenshots about the feature request here.

@moonD4rk moonD4rk added the enhancement New feature or request label Jan 30, 2024
@rowingdude
Copy link

Hello, I would like to take a stab at this.

I'd like to propose using a single map and a key of the folder.

First, we modify the original "walk" function to get rid of the if statements which cause obscurity. This eliminates one map and established a directory of composite keys, which are much faster to look up than re-traversing.

func chromiumWalkFunc(items []item.Item, itemPaths map[string]map[item.Item]string) filepath.WalkFunc {
	return func(path string, info fs.FileInfo, err error) error {
		if !isValidItem(path, info, items) {
			return nil
		}
		
		profileFolder := fileutil.ParentBaseDir(path)
		if strings.Contains(filepath.ToSlash(path), "/Network/Cookies") {
			profileFolder = fileutil.BaseDir(strings.ReplaceAll(filepath.ToSlash(path), "/Network/Cookies", ""))
		}
	
		for _, v := range items {
			if info.Name() == v.Filename() {
				itemPaths[composeKey(profileFolder, v)] = path
				break 
			}
		}
		
		return nil
	}
}
func composeKey(profileFolder string, item item.Item) string {
	return fmt.Sprintf("%s|%s", profileFolder, item.Filename())
}

We then write up a function to test if the item is what we're looking for:

func isValidItem(path string, info fs.FileInfo, items []item.Item) bool {
	if info.IsDir() || strings.Contains(path, "System Profile") {
		return false
	}
	for _, v := range items {
		if info.Name() == v.Filename() {
			return true
		}
	}
	return false
}

I have not bench marked this yet but present it here as a possible alternative to your original idea.

@moonD4rk
Copy link
Owner Author

Thanks for your suggestion @rowingdude, The following structure is defined in the new version. Use the KeyFile of Chromium to determine the location of RootPath .

There are still some bugs that need to be fixed. I will submit the code in the next few days and look forward to your review at that time.

type BrowserProfile struct {
	// Name name like 'Default', 'BrowserProfile 1', etc.
	Name string
	// Path to the master key file (e.g., 'Local State', 'key4.db',
	KeyPath string
	// MasterKey is the key used to decrypt the browser data.
	MasterKey []byte
	// Map of data types to file paths
	DataFiles map[types.DataType][]string
}

func (bd *BrowserData) findChromiumProfiles(rootPath string, dataTypes []types.DataType) (map[string]*BrowserProfile, error) {
	profiles := make(map[string]*BrowserProfile)
	var sharedKeyPath string
	err := filepath.WalkDir(rootPath, func(path string, entry fs.DirEntry, err error) error {
		if err != nil {
			return err
		}
		normalizedPath := filepath.ToSlash(path)
		// Skip directories that should not be included
		if entry.IsDir() && skipDirs(normalizedPath, defaultExcludeDirs) {
			return fs.SkipDir
		}
		for _, dataType := range dataTypes {
			if entry.Name() != dataType.Filename() {
				continue
			}
			// Calculate relative path from Chrome baseDir path
			profileName := extractProfileNameFromPath(rootPath, path)
			// handle shard key
			if profileName == "" && dataType == types.ChromiumKey {
				sharedKeyPath = path
			}
			if profileName == "" {
				continue
			}
			if _, exists := profiles[profileName]; !exists {
				profiles[profileName] = newProfile(profileName)
				profiles[profileName].DataFiles[dataType] = []string{path}
			} else {
				profiles[profileName].DataFiles[dataType] = append(profiles[profileName].DataFiles[dataType], path)
			}
		}
		return nil
	})
	// Assign the shared key path to all profileNames
	for _, profile := range profiles {
		profile.KeyPath = sharedKeyPath
	}
	return profiles, err
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants