YouTube Video to Tabs and Lyrics

(github.com)

135 points | by sixall 263 days ago

24 comments

  • rwl4 263 days ago
    So I'm trying to understand. Is this spam for the Lamucal service? I saw this same code posted on Reddit the other day under a different name. Here are a few repos with the exact same code under different names:

    - https://github.com/DoMusic/Hybrid-Net

    - https://github.com/TuneMusic/NiceMusic

    - https://github.com/JoinMusic/fish

    - https://github.com/Famuse/CombineNet

    - https://github.com/AIAudioLab/AITabs

    - https://github.com/AIMusicLab/MicroMuisc

    I'm pretty sure there are more, but I'll stop there. Especially suspicious considering all the usernames.

    Here's a post from yesterday on Reddit:

    - https://www.reddit.com/r/coolaitools/comments/1ervthn/found_...

    I'm guessing the general process here is:

    - Push novelty (but unusable to most people) code to new Github repo

    - Submit that code to Reddit/Hacker News

    - People see it and are impressed by the novelty code, despite not running it due to missing the models themselves, etc. They upvote and subscribe ($$$) to actually try it.

    - Repeat

    I understand the desire to promote one's new service, and the product seems like it could be interesting, but this is not the way to get the word out. Reputation matters.

    Edit:

    Check out the user deeplover's post/comment history. One submission with the MicroMusic (see above) repo, and one comment, see below.

    Also, the post by user liwei0517 is almost exactly like BigOrange688 on Reddit. See: https://www.reddit.com/r/MachineLearning/comments/1es0deh/co...

    • skeaker 263 days ago
      This is really strange. I'd rather not name names in case I'm wrong, but some of the comments in this thread that link to "things they made" on Lamucal are by very fishy accounts. Account was made a few months ago, only has four comments on seemingly random articles, the comments are either extremely brief or seemingly ChatGPT-generated, then after months of inactivity they pop up in this thread. Not usually one to complain about this but with your comment showing all the other times this has popped up it's way too strange.
    • sixall 263 days ago
      I am an entrepreneur, and my startup team is about to fail. In response to the questions raised by rwl4, I am very sorry. All the features on our website are completely free for everyone to use. If we can help some people, it will be a small consolation for the team before the failure. I apologize again.
      • rwl4 263 days ago
        Look, I get it. Startups are hard. Desperation can lead to poor choices. But spamming isn't the answer and will burn bridges faster than you can build them.

        If you are genuinely sorry and want to (possibly) turn this around, here are some ideas:

        1. Consolidate your work into one solid well documented Github repo, including a small toy model that people can actually use and play with.

        2. Write a blog post or tutorial about your tech. That'll show your expertise without making people feel like they are being duped.

        3. Engage in relevant communities genuinely. Build relationships, not just a user base.

        4. Use proper channels for promotion... Show HN exists for a reason.

        Remember, reputation is currency in this industry. It's hard to earn and easy to lose. Take the long view, build something good, be transparent, and let organic interest do its thing...

        If your product is actually solid, there are ways to get it out there. It might be slower, but it's at least sustainable. Good luck.

        • sixall 263 days ago
          Thank you very much, my friend. You are a good person.
    • HanClinto 263 days ago
      This comment is a far more interesting story than the project itself.
    • crustaceansoup 263 days ago
      That other, probably-related Hacker News user posted a response to a suicide crisis post that looks like a "four-paragraph standard ChatGPT response", complete with a "Lastly, " at the end. I hope that wasn't done just to increase that account's seeming legitimacy because wow, what incredibly bad taste.
    • sixall 263 days ago
      As an entrepreneur, doing this kind of not-so-glorious thing to promote my own features, I already feel quite pathetic about myself. Being pointed out by rwl4, I can also take a deep breath. I would like to apologize to the people who have been disturbed, and also apologize to the platform.
      • jokethrowaway 262 days ago
        Cool tech but right now making money with AI is 10x harder than raising from investors.

        I'd recommend consulting on the side to keep lights on, there is demand for AI consulting, while there is still hype about AI.

      • hu3 263 days ago
        I respect the hustle.

        As an entrepreneur, you're light years further than most of us will ever be.

        Chin up! Launch again, launch different, launch something else, Pivot if need be. You make me proud.

  • rrherr 263 days ago
    Here's the most impressive results I've seen for automated guitar transcription:

    High-resolution guitar transcription via domain adaptation

    Demo Videos: https://xavriley.github.io/HighResolutionGuitarTranscription... Paper: https://arxiv.org/abs/2402.15258

    > We propose the use of a high-resolution piano transcription model to train a new guitar transcription model. The resulting model obtains state-of-the-art transcription results on GuitarSet in a zero-shot context, improving on previously published methods.

    • bertili 263 days ago
      The MIDI Guitar software [1] does this in real-time, similar to Rolands hexaphonic pickups, but with a standard guitar signal.

      MIDI Guitar 3 is in open beta testing [2] and adds MPE (multidimensional polyphonic expression) MIDI.

      [1] jamorigin.com [2] jamorigin.com/beta

    • senbrow 263 days ago
      Anything as good as this for drum tab transcription?
      • rrherr 263 days ago
        This isn't exactly what you asked for, but there's a "drumsep" model, which takes a drum audio track and separates it into 6 stems: kick, snare, toms, hi-hat, ride, and crash.

        Ctrl+F for "drumsep" in this doc:

        Instrumental, vocal & other stems separation & mix/master guide - UVR/MDX/Demucs/GSEP & others - Google Docs https://docs.google.com/document/d/17fjNvJzj8ZGSer7c7OFe_CNf...

        • xavriley 263 days ago
          I’m the author of the high resolution guitar model posted in a comment above. I have a drum transcription model that I’m getting ready for release soon which should be state of the art for this. I’ll try to update this thread when I’m done
  • kranner 263 days ago
    The "tabs" seem to be arpeggiations of the chords, which might have been some use if the chord detection had worked well, which doesn't seem the case. I see chords and tabs being generated from sections which have only spoken audio, while actual guitar parts are not notated at all. The arpeggios are not consistent either and switch arbitrarily to upstrokes/downstrokes and back to arpeggios.

    edit: removed a reference to a competing product

    • LordShredda 263 days ago
      re: edit Can I ask for said competing product? I really like the idea and want to see how others do it
      • kranner 263 days ago
        Chordify (web) and Capo (Mac/iOS) but these only do chord detection.
  • criddell 263 days ago
    I tried to get it to generate tabs for Where is my Mind by the Pixies. I see the chords, but get the NO icon (red circle with diagonal bar) when I try to click on tabs. Am I doing something wrong?

    A couple of weeks ago I asked one of the AIs to teach me this song. It responded that it can't teach specifics or tell me strumming patterns because it would be a copyright violation. I told it that if I went to a human teacher, they would have no problem teaching me how to play along to the song. That was a good enough argument to get the AI to changed its mind (whatever that means) and produced a chord chart and strumming pattern (which was wrong).

    • saaaaaam 263 days ago
      It’s a weird technicality of copyright law. When a human teacher plays a song that is a private performance of a composition. When a computer plays a song copyright law treats this as a mechanical copy of the composition and public performance of the composition. When a computer displays musical notation for a composition that is also a mechanical copy.

      Private performance of a composition is permitted under copyright law. Creating a mechanical copy and making a public performance requires a licence from a copyright owner and so without that licence there is, in the eyes of copyright law, a breach of copyright.

    • thekevan 263 days ago
      Off topic, but you may enjoy this guy's cover and hopefully get some pointers on how to play it as well. Although he definitely plays it differently than most

      https://www.youtube.com/watch?v=JhjA2nvVD7U

  • authorfly 263 days ago
    Nice idea, but gives the wrong chords for jazz music: https://lamucal.com/chords/emmet-cohen/after-youve-gone-patr...

    E.g. Bb instead of Ebmaj7

    Bb7 instead of Bm7b5

  • neilyio 263 days ago
    Very happy to see more tools like this. There is so much potential for interactive tabs and sheet music with YouTube videos.

    I only found out about https://www.soundslice.com recently. I'm not sure how it managed to evade me for years of searching for music resources on the internet... but for anyone interested in sheet music, I can't recommend it enough.

    The design of the whole platform is so minimal and beautiful, and having notation synchronized with YouTube is simply brilliant. Built by one of the co-creators of Django, too!

    • adrianh 263 days ago
      Thanks — glad you found our site and like it! :-)
      • PretzelPirate 263 days ago
        Have you considered adding note detection and note-by-note playing of tabs?

        I use that in Rocksmith+, but their detection isn't great and neither is their interface. I'd prefer to use something like Sound Slice with that feature.

  • dadver 263 days ago
    I played around a few minutes with the various features. The voice removing was kind of impressive, though I don't know how novel that is.

    I tried making some AI covers, too, which was kind of fun. For one of my tries, I submitted Nirvana's Smells Like Teen Spirit for AI voice generation to make a cover of Carola's song "Främling" (Sweden's ESC song 1983 which came in third, a very non-Nirvana-type of song). At first, I thought the voice sounded pretty much like a Swedish Kurt Cobain, then the more I listened to it, all I could hear was the Swedish artist Nordman, and it dawned upon me that they have similar voice styles. I tried lowering the pitch, and then I was certain I recognised the voice from another artist but couldn't place it. So I'm leaning towards the AI voices being trained on some not-so-unfamiliar artists rather than there being some cool AI magic happening, though I'm out of my depth here.

  • sixall 263 days ago
    I am an entrepreneur, and my startup team is about to fail. In response to the questions raised by rwl4, I am very sorry. All the features on our website are completely free for everyone to use. If we can help some people, it will be a small consolation for the team before the failure. I apologize again.
  • buildsjets 263 days ago
    Pretty cool! Is there a way to either detect or enforce alternate tunings? There is a world beyond EADGBe... I put in a few songs that I know of which have trivial chord fingerings in drop D, and it comes up with some correct but convoluted chord fingerings in standard tuning.
  • SoftMachine 263 days ago
    I don't often read hackernews for Lamucal AI spams but when I do, it's always nice.
  • ksr 263 days ago
    I'm looking to do the opposite: Given a melody in MusicXML / MIDI, generate an accompaniment "in the style of". Any pointers?
  • riiii 263 days ago
    Very interesting. I don't suppose this would work with instrumental music?

    Anyone know of a thing that does?

  • smrtinsert 263 days ago
    This is going to save me a lot of time for when I actually have time to play music. Much better than whipping out Audacity and its chord estimation, manually grabbing a video etc.
  • TrackerFF 263 days ago
    Seems like a cool concept, but the tab function was more or less useless. Tried a bunch of different songs in various complexities, couldn't get anything convincing.
  • ndriscoll 263 days ago
    Neat, it would also be awesome to package it into something like a Clone Hero/Rocksmith tool/plugin to generate charts just like Audiosurf did.
  • zerop 263 days ago
    Off topic: what's best way to generate good quality videos given a transcript. Only automation, no manual work. I can code.
  • CMLab 263 days ago
    AI cover song platforms, How to address issues related to copyright, legal, and ethical concerns?
    • sixall 263 days ago
      Thank you for your concern! We take copyright and legal issues very seriously. Our platform employs strict copyright protection measures to ensure that all generated content complies with relevant laws and regulations. We use technical means to prevent unauthorized use and encourage users to respect original works. If you have any copyright-related questions or concerns, please feel free to contact us, and we will address them promptly.
  • guitarlimeo 263 days ago
    I was expecting something like this https://www.youtube.com/watch?v=nLtlyzWuoqM

    but was somewhat disappointed. The site is cool, can give you a headstart when transposing a song to practice, but the chords were quite off in a few examples I tried.

  • afpx 263 days ago
    Wow - nice job!
  • deeplover 263 days ago
    [dead]
    • yorwba 263 days ago
      Aren't those completely different? As rwl4 pointed out, your only submission was also about this product, so probably you're trying to promote it, but shouldn't you at least pick a case where it works well?
  • EAZR4400 263 days ago
    [flagged]
  • joffendi 263 days ago
    [flagged]
    • liwei0517 263 days ago
      It is indeed quite interesting .
    • riiii 263 days ago
      A convicted felon.
  • liwei0517 263 days ago
    I just trained a Taylor Swift voice model using Lamucal ( https://lamucal.com/ai-cover/share/66be2087bc3fdb000baf3cac ),

    The mid-range is eerily close to Taylor's voice - there were moments when I almost thought it was really her singing. But, If you listen closely, you can still catch a tiny hint of that robotic sound.

    A Bar Song : (Taylor Swift Cover) https://lamucal.com/ai-cover/song-share/66be244cbc3fdb000e76... Original: A Bar Song (Tipsy) https://www.youtube.com/watch?v=t7bQwwqW-Hc