How to search for compound words, and get the word parts returned

Question

I need to be able to search for compound words like "menneskerettighedsforkæmperens" (the human rights activist's) and find the words "menneske" (human), "rettighed" (right) and "forkæmper" (activist).

I have all the words (and hundreds of thousands of other words) listed in MySQL, all in separate rows with additional information about each word (like hyphenation, which is what I need the DB to return), but I need to search a lot of words at the same time, which can be painfully slow.

I'm currently using a MySQL database, but willing to switch it out for something better suiting my needs, maybe some kind of NoSQL or elasticsearch. But I haven't been able to find any examples of how to accomplish what I'm looking for in any other type of databases. So if anyone can help me out, I would really appreciate it.

Rick James · Answer

I would look for a library that specializes in splitting compound words.  Search Java, PHP, Perl, etc.  SQL is not likely to be involved.

If you must use a database, then consider this for a piece of the algorithm:

SELECT word FROM words
    WHERE word < 'menneskerettighedsforkæmperens'
    ORDER BY word DESC
    LIMIT 5;

That is likely to return a few compound words that begin with 'menneske' and perhaps 'menneske', itself.

In MySQL, that query will be quite fast (assuming an index on word).  You would need some code in addition to a few SQL calls.

How to search for compound words, and get the word parts returned

One Answer

Add your own answers!

Related Questions

Oracle 11gR2 – How To Audit SYS.USER$ Table

Common entities with differing attributes across multiple databases

SqlPackage does not pick up variables from profile

This simple mysql query is taking 2-3 seconds

Powershell Invoke-Sqlcmd Replacing £ with?

how to nest query with avg and sum group by outer in MySQL 5.7

Something is totally wrong with my query. .. Slow. .. Why?

PostgreSQL pg_restore: How do I save script report to a txt file?

How to both reference and return a result set from a stored procedure in mysql?

Upgrade Postgres 9.6 to 12

Postgres normalize human name string

Stored procedure for adding cascade

Using hard-coded host name in logon trigger

MySQL Optimization Guidance Needed

MySQL subquery with WHERE filters not working as expected

Sybase IQ cache database result

In what order does sqllogship.exe process groups of log files?

Help Outputing MySQL General Log to Named Pipe

How can I capture error thrown by SET_LOCKTIMEOUT in ola Hallengren

Windows Azure Table Row Key strategy, what makes a useful RowKey?

Ask a Question