Melatonin is generally safe to dogs, but xylitol toxicity can be quite serious.  Like any other toxin, it depends on the dose though.  I would recommend calling the ASPCA poison control straight away as they will be able to help you ensure you get the appropriate treatment.  Also, if you get a case number from them and you need to go the veterinarian, the veterinarian can reference the case to ensure they get the proper detox protocol.